SSH key authentication is one of the most practical ways to secure remote access to a Linux server. Instead of entering an account password every time you connect, OpenSSH verifies your identity using a cryptographic key pair.
For anyone managing VPS instances, cloud infrastructure, or production servers, setting up SSH key authentication is an important step toward reducing unnecessary security risks.
However, generating an SSH key is only part of the process. The server must also be configured to recognize the public key and allow authentication through it.
In this guide, we’ll walk through how to configure SSH key authentication on Ubuntu 24.04, verify that everything works correctly, and optionally disable password-based login once key authentication has been tested.
What Is SSH Key Authentication?
SSH key authentication is an authentication method that uses two cryptographic keys: a public key and a private key.
The public key is stored on the remote server, while the private key stays on the computer you use to connect.
When an SSH connection is established, the server checks whether the connecting client can prove ownership of a trusted private key.
Unlike traditional password authentication, the private key itself is never sent to the server.
This approach offers several advantages:
- Reduces exposure to password-guessing attacks.
- Allows secure authentication without repeatedly entering account passwords.
- Supports passphrase-protected private keys.
- Makes server access easier to manage across multiple machines.
- Provides a foundation for more advanced SSH security configurations.
If you haven’t generated an SSH key pair yet, follow our tutorial on How to Generate and Use SSH Keys on Ubuntu before continuing.
Prerequisites
Before configuring SSH key authentication, make sure you have:
- An Ubuntu 24.04 server.
- OpenSSH Server installed and running.
- A user account with sudo privileges.
- An SSH key pair generated on your local computer.
- Existing access to the server through SSH.
If OpenSSH Server is not installed yet, you can follow our guide on How to Install OpenSSH Server on Ubuntu 24.04.
For this tutorial, we’ll use the following example:
| Setting | Example |
|---|---|
| Server IP | 192.0.2.10 |
| SSH username | ubuntu |
| SSH port | 22 |
| Key algorithm | Ed25519 |
| Operating system | Ubuntu 24.04 |
Replace these example values with your actual server details.
Step 1: Verify That Your SSH Public Key Is Available
First, check that your local machine already has an SSH key pair.
On your local computer, run:
ls -la ~/.ssh/You should see files similar to:
id_ed25519
id_ed25519.pubThe id_ed25519 file is your private key, while id_ed25519.pub contains your public key.
Never upload or share your private key with anyone. Only the public key should be transferred to the server.
If you use a different key filename, make sure to use the correct path in the following steps.
Step 2: Copy Your Public Key to the Ubuntu Server
The easiest way to install your public key on the server is by using ssh-copy-id.
From your local computer, execute:
ssh-copy-id [email protected]If your SSH server uses a custom port, specify it with the -p option:
ssh-copy-id -p 2222 [email protected]You’ll be prompted to enter your existing server account password.
Once authenticated, ssh-copy-id automatically creates the required SSH directory if necessary and adds your public key to the server’s authorized_keys file.
By default, the file is located at:
/home/ubuntu/.ssh/authorized_keysYou can verify that the public key has been added by logging in to the server and running:
cat ~/.ssh/authorized_keysYou should see your public key listed in the output.
Each authorized public key normally occupies its own line.
Step 3: Configure SSH Directory and File Permissions
Incorrect file permissions are one of the most common reasons SSH key authentication fails.
OpenSSH checks the ownership and permissions of important SSH files before accepting authentication requests.
Connect to your Ubuntu server and execute:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keysThese commands apply the following permissions:
| File or directory | Permission | Purpose |
|---|---|---|
~/.ssh | 700 | Accessible only by the user |
authorized_keys | 600 | Readable and writable only by the user |
Also make sure the files belong to the correct user:
chown -R "$USER":"$(id -gn)" ~/.sshIf the user’s home directory has overly permissive write permissions, OpenSSH may also reject authentication.
Check it with:
ls -ld ~ ~/.ssh
ls -l ~/.ssh/authorized_keysThe important thing is that other users must not be able to modify the SSH authentication files.
Step 4: Enable Public Key Authentication in OpenSSH
Now that the public key is installed, it’s time to check the SSH server configuration.
Ubuntu uses the following configuration file:
/etc/ssh/sshd_configBefore making changes, create a backup:
sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backupOpen the configuration file using your preferred text editor:
sudo nano /etc/ssh/sshd_configFind the following directive:
#PubkeyAuthentication yesMake sure it is configured as:
PubkeyAuthentication yesThis explicitly enables public key authentication.
Also check the following directive:
AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2This tells OpenSSH where to look for authorized public keys.
For most standard Ubuntu installations, the default location is sufficient, so there is usually no need to change it.
Important: Ubuntu may also load additional SSH configuration files from /etc/ssh/sshd_config.d/. If a setting does not behave as expected, inspect those files as well. Configuration directives from included files can affect the final server configuration.
You can check the effective public key authentication setting with:
sudo sshd -T | grep -i pubkeyauthenticationExpected output:
pubkeyauthentication yesStep 5: Test SSH Key Authentication
Before disabling password authentication, make sure you can successfully connect using your private key.
Open a new terminal window on your local computer and run:
ssh -i ~/.ssh/id_ed25519 [email protected]If your private key is already loaded into your SSH agent or uses the default filename, you may also connect with:
ssh [email protected]If everything is configured correctly, you should be able to access the server without entering your account password.
You may still be asked for your private key’s passphrase. This is normal and does not mean SSH password authentication is being used.
For additional verification, enable verbose SSH output:
ssh -vvv -i ~/.ssh/id_ed25519 [email protected]Look for a message similar to:
Authenticated to 192.0.2.10 using "publickey".This confirms that the connection was authenticated using your SSH key.
Step 6: Disable Password Authentication (Optional)
Once SSH key authentication is working reliably, you can disable password-based login to further reduce the server’s attack surface.
This is particularly useful for internet-facing VPS instances that receive frequent automated login attempts.
Edit the SSH configuration:
sudo nano /etc/ssh/sshd_configFind the following directive:
#PasswordAuthentication yesChange it to:
PasswordAuthentication noYou should also check keyboard-interactive authentication:
KbdInteractiveAuthentication noThese settings prevent ordinary password-based and keyboard-interactive authentication methods from being used.
A word of caution: Do not disable password authentication until you have successfully tested SSH key login in a separate terminal session. Keep your existing SSH session open while applying and testing the changes.
Also make sure you have another recovery method, such as your VPS provider’s web console, in case you accidentally lock yourself out.
Step 7: Validate and Reload the SSH Service
Before applying any SSH configuration changes, validate the configuration syntax.
Run:
sudo sshd -tIf the command returns no output, the configuration syntax is valid.
You can then reload the SSH service:
sudo systemctl reload sshCheck the service status:
sudo systemctl status sshYou should see an active service status.
On Ubuntu, the systemd service is commonly named ssh, even though the underlying daemon is sshd.
Now open another terminal window and test the connection again:
ssh -i ~/.ssh/id_ed25519 [email protected]If you can connect successfully, your SSH key authentication configuration is working as expected.
A Practical Troubleshooting Scenario
One situation that can be confusing when configuring SSH key authentication is when everything appears correct, but the server continues asking for the account password.
The public key is already present in authorized_keys, the private key is available on the client, and the SSH service is running normally. Yet the connection still falls back to password authentication.
In practice, this is often caused by something small rather than a major OpenSSH problem.
For example, the .ssh directory might have incorrect permissions, the public key might have been added to the wrong user’s home directory, or an additional configuration file might be overriding the expected SSH settings.
When troubleshooting this type of issue, I prefer checking things in a specific order rather than immediately restarting the SSH service.
Start by checking file ownership and permissions:
ls -ld ~/.ssh
ls -l ~/.ssh/authorized_keysThen inspect the effective SSH configuration:
sudo sshd -T | grep -E 'pubkeyauthentication|passwordauthentication|authorizedkeysfile'Finally, monitor the SSH authentication logs while attempting a new connection:
sudo journalctl -fu ssh.serviceOn Ubuntu, these logs can reveal useful details about rejected keys, incorrect permissions, and authentication failures.
This approach makes troubleshooting much easier because you can identify whether the issue comes from the client, the user’s SSH files, or the server configuration.
Common SSH Key Authentication Problems
Even when the configuration looks correct, a few common issues can prevent SSH key authentication from working.
1. Permission Denied (Publickey)
This usually means the server cannot accept the public key presented by the client.
Check the following:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keysAlso verify that the public key belongs to the account you’re trying to access.
2. SSH Still Requests an Account Password
There are several possible reasons:
- Public key authentication is disabled.
- The public key is stored in the wrong location.
- The SSH configuration contains conflicting directives.
- File ownership or permissions are incorrect.
- The client is offering a different private key.
Use verbose mode to investigate:
ssh -vvv [email protected]You can also inspect the effective SSH server configuration using sshd -T.
3. Too Many Authentication Failures
This can happen when your SSH agent contains several keys and the server rejects too many attempts before reaching the correct one.
Specify the intended private key explicitly:
ssh -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519 [email protected]4. SSH Service Fails After Configuration Changes
If the SSH service does not reload successfully after editing the configuration, check the syntax first:
sudo sshd -tThen inspect the service logs:
sudo journalctl -u ssh.service -n 50 --no-pagerFor a more detailed guide to connection-related problems, see our tutorial on How to Fix SSH Connection Refused.
Final Thoughts
Configuring SSH key authentication on Ubuntu 24.04 is a relatively straightforward process, but the details matter.
Installing the correct public key, setting appropriate file permissions, checking the OpenSSH configuration, and testing the connection before disabling password authentication can make a significant difference.
For administrators managing multiple Linux servers, key-based authentication is also easier to maintain than relying on individual account passwords.
Take the time to verify your configuration before closing your existing SSH session. A few minutes of testing can save you from an unnecessary server recovery situation later.
Frequently Asked Questions
1. Is SSH key authentication more secure than password authentication?
Generally, yes. SSH key authentication uses cryptographic key pairs rather than relying on passwords that can be guessed or reused. Protecting your private key with a strong passphrase adds another layer of security.
2. Do I need to disable password authentication after configuring SSH keys?
No. It is optional. However, disabling password authentication after successfully testing key-based access can reduce exposure to automated password-guessing attacks.
3. Where are SSH authorized keys stored in Ubuntu?
By default, OpenSSH checks the following location:
~/.ssh/authorized_keysThe file contains the public keys permitted to authenticate as that particular user.
4. Why does SSH still ask for a password after installing my public key?
This can happen because of incorrect file permissions, an incorrect username, a missing public key, or server configuration issues. Running SSH in verbose mode with ssh -vvv can help identify the cause.
5. Can I use multiple SSH keys for one Ubuntu server?
Yes. You can add multiple public keys to the same authorized_keys file. Each key should be placed on a separate line.
6. Does SSH key authentication completely eliminate the need for passwords?
Not necessarily. Your server account may still have a password for local authentication or other administrative purposes. Disabling SSH password authentication only prevents that authentication method from being used for SSH connections.
7. How do I check whether SSH key authentication is enabled?
Run this command on your Ubuntu server:
sudo sshd -T | grep pubkeyauthenticationIf the output is pubkeyauthentication yes, public key authentication is enabled in the effective SSH server configuration.
This tutorial was tested against standard OpenSSH configuration practices for Ubuntu 24.04. Configuration details may vary depending on additional SSH settings and server provisioning tools.