Securing SSH on Linux Part 2

In this part I will explain how to further improve the server’s security by replacing traditional passwords with public key authentication.

Before deciding to follow this tutorial please consider the following problem: While replacing passwords with private/public key pairs is an enormous improvement in security it also requires specially preparing your workstation for logging into the server, backing up your key pair and ensuring that the private key does not fall into the wrong hands.

Note: I strongly suggest that you complete part 1 of this tutorial first.

Public / private key authentication is a subject that can be rather counter-intuitive and in all honesty my understanding of the mathematical basis is superficial to say the least. The important thing to understand is you have a matching pair of non-identical files. One (the private key) for proving identity, the other (a public key) for verifying identity. The one is on the user’s PC computer the second is copied onto the VPS.

Generate the key files:

user@localhost:~ $ ssh-keygen -t rsa -f ~/.ssh/remotehost

Replacing  remotehost with the name of your VPS. Be sure to use a nice long password.

Upload the public key onto your server:

user@localhost:~ $ scp ~/.ssh/

Log into the server and add the key into the authorized keys list:

user@localhost:~ $ ssh admin@remotehost
admin@remotehost:~ $ cat ~/.ssh/ >> ~/.ssh/authorized_keys

Configure SSH to allow but not require authenticating with public keys. This way you can still login if you haven’t enabled properly. Open up the SSH Daemon configuration file as root:

user@remotehost:~ $ sudo vi /etc/ssh/sshd_config

The commands that this file needs to contain are:

  • RSAAuthentication yes
  • PubkeyAuthentication yes
  • AuthorizedKeysFile .ssh/authorized_keys

Be sure not to leave contradictory lines in the configuration file.

At this point you will have to restart the SSH Daemon to put the changes into effect. Exactly how this is done depends on what Linux distribution. The command for this is either:

user@remotehost:~ $ sudo service sshd restart


user@remotehost:~ $ sudo systemctl restart sshd

Where sshd may be replaced with, ssh, open-ssh or open-sshd.

Now you have to edit your local user SSH configuration. On your local machine edit the ~/.ssh/config file and add the following stanza:

Host nickname
IdentityFile ~/.ssh/remotehost_rsa
User admin
Port 22
Hostname remotehost

Where nickname is a shortened name for your VPS.

Test that everything so far works by logging in with the key:

user@remotehost:~ $ ssh nickname

In the event that remotehost hasn’t been correctly configured your can simply log in the way that you have previously been using.

Finally log back in, open up ~/.ssh/config back up (using sudo) and edit it to contain:

PasswordAuthentication no

You have now completely secured yourself from SSH password guessing. Remember to back up your private key in case your PC fails and be prepared to configure any additional PC’s for accessing the server.

By Ryan McCoskrie

I am a nerd trapped in the country side of North Canterbury, New Zealand trying to get an IT-related career off of the ground. Legal disclaimer: Ryan McCoskrie is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to I'm also signed up with Fishpond but they aren't as strict about this stuff.

Leave a comment