How to Enable Private Key Authentication over SSH on Linux

In an earlier article, we saw how to disable root logins. In that, we obtain root access by first logging in as another user, and then getting root permissions using the “su – root” command. But there’s another secure way to get root access without the hassle of first logging in as someone else. And that’s using private key authentication. Most tutorials on using private keys assume some kind of prior setup or GUI interface using which you can set up authentication. In this tutorial, I’ll show you how to do this entirely over the command line using a Windows client. So let’s get started.

First, I’m assuming you have the following already set up:

1. Working root access
2. SSH enabled using PuTTY for Windows
3. Knowledge of how to edit files on a Linux server

Step 1: Temporarily Enable Root Login

If you’re NOT using private key authentication for the root user, then skip this step. On the other hand, if you want to log in as root using private keys, then  it’s necessary. We’re going to need root login to download the private key using the “pscp.exe” program. So go into the following file:

/etc/ssh/sshd_config

And add back the “#” from the beginning of this line:

PermitRootLogin no

So that it becomes:

#PermitRootLogin no

This will allow us to login to root with a password. Don’t worry! We’ll change it back to something more secure in a later step.

Step 2: Generate Public/Private Key Pairs

Now it’s time to generate the public/private key pairs on the server. Note that I’m assuming you have a Windows client and a Linux server. So the following commands are to be executed on the server – not the client! Enter the following when logged in as root:

ssh-keygen -t dsa

The “-t” option is used to specify the encryption format. I’ve chosen to use DSA here.

This will give you a default location where the private and public keys are to be saved. Press “Enter” to accept the default locations. You can also type a passphrase for added security if you want. If not, leave it blank. The output will look like this:

These keys will be linked to the account used to create them. So if you’re logged in as root, they are specific to the root user. The keys by default are stored in the following folder:

/root/.ssh/

There are two files here:

1. id_dsa
2. id_dsa.pub

The first is your private key which we need to download. The second is a public key residing on your server.

Step 3: Authorizing the Public Key

Just because you’ve generated a public key doesn’t mean that you can start using it immediately. You need to “authorize” it first. In the /root/.ssh/ folder, there is a file called “authorized_keys” and we need to append our public key to it. First, navigate to the folder by typing in the following:

cd /root/.ssh

And then append the public key like this:

cat id_dsa.pub >> ~/.ssh/authorized_keys

Like so:

Without this step, you’ll get an error message saying “server refused our key” when you try and use it. So now we’ve authorized our public key to be used to log into the server.

Step 4: Download the Private Key onto your Local Client

While the public key resides on the server, we need to transfer the private key onto our local PC. By default, PuTTY doesn’t allow us to do this. So we use a complimentary program called “pscp” instead. Visit the PuTTY download page and obtain the pscp.exe file as shown here:

Now open up a Windows command line at the location where you’ve downloaded it and type in the following:

pscp root@XXX.XXX.XXX.XXX:/root/.ssh/id_dsa C:\private_key

Replace XXX.XXX.XXX.XXX in the above line with the IP address or domain name of your server. And replace “private_key” with a folder name of your choice in C:\ on your Windows PC. Or simply create a folder called “private_key”. The pscp program will then ask you for your root password so that it can go ahead with the transfer as shown here:

This is the reason for “Step 1” at the beginning of this tutorial. Without it, you wouldn’t be able to use your password to log into root and download the private key! Once the private key is downloaded, it should be available in the specified folder as shown here:

Step 5: Disable Password Root Login. Enable Private Keys

Again, if you’re not interested in root login with private keys, then ignore this step just like you ignored Step 1.

Now we can revert Step 1 and disable root login again. However, this time we don’t want to disable all root logins. What we want to do is to disable root login only via passwords, and yet allow private key logins. To do that, change the line to the following instead:

PermitRootLogin without-password

Note that you need to remove the “#” symbol at the beginning of the line as in the previous step. Once that’s done, save your changes and restart sshd via the following command:

systemctl restart sshd

Root access via password is now disabled. But we can still connect via private keys. If you omit this step, you’ll get an error message like this when you try and connect “Server refused public-key signature despite accepting key!”

Step 6: Convert the Private Key to a PuTTY File

The file we just downloaded won’t work with PuTTY. So we have to convert it using a program called PuTTYgen from the same PuTTY download page. Download the file and run it on Windows (You might get a warning – ignore it and run the file anyway).

Click the “Load” button and browse to the private key file you just obtained from the server: (Click “All Files” from the dropdown box to see it).

Now we need to generate a PuTTY file. Since we used the DSA protocol to generate the key, change the selection at the bottom radio button to “SSH-2 DSA” instead, and click “Save private key”:

This will place a PuTTY (*.ppk) file in your key folder. We’re now going to use this to login to our server.

Step 7: Login to the Server Using the Private Key and PuTTY

To login directly to the server using our private key, open up PuTTY as usual and type in your IP address/domain name and port as usual. But now, navigate to the “Connection” entry on the left hand side and click “Data” as shown here:

In the field called “Auto-login username”, type the name of the account you generated the public/private keys for. In this article, I’m doing it for root, so I write “root” as shown above.

Next, expand the “SSH” entry and click “Auth”:

Hit “Browse” on the right hand side and choose the PuTTY (.ppk) file we generated in the previous step. Finally, head back to the main “Session” screen at the top and under “Saved Sessions”, type a name for the connection and hit “Save”. So the next time you want to log in, you just need to double click the entry of the box to log into your server. Opening the connection this way automatically logs us into the server like this:

And that’s it! You can now log in as any user with a single click using a private key with the process described above. Keep in mind that to login as root, you need to set the “PermitRootLogin” line to “without-password” in /etc/ssh/sshd_config” as shown in Step 5.