You want only certain accounts on your machine to accept incoming SSH connections.
Use sshd 's AllowUsers keyword in /etc/ssh/sshd_config. For example, to permit SSH connections from anywhere to access the smith and jones accounts, but no other accounts:
/etc/ssh/sshd_config: AllowUsers smith jones
To allow SSH connections from remote.example.com to the smith account, but no other incoming SSH connections:
Note this does not say anything about the remote user "email@example.com." It is a rule about connections from the site remote.example.com to your local smith account.
After modifying sshd_config, restart sshd to incorporate your changes.
AllowUsers specifies a list of local accounts that may accept SSH connections. The list is definitive: any account not listed cannot receive SSH connections.
The second form of the syntax (user@host) looks unfortunately like an email address, or a reference to a remote user, but it is no such thing. The line:
means "allow the remote system called remotehost to connect via SSH to my local account user."
A listing in the AllowUsers line does not guarantee acceptance by sshd: the remote user must still authenticate through normal means (password, public key, etc.), not to mention passing any other roadblocks on the way (firewall rules, etc.).