How to Generate and Use SSH Keys on Ubuntu

How to Generate and Use SSH Keys on Ubuntu

by

Connecting to a remote Linux server using SSH is something most system administrators do almost every day. Whether you’re managing a VPS, deploying an application, or troubleshooting a production environment, SSH provides a reliable way to access remote machines.

However, entering a password every time you connect can become inconvenient, especially when you’re managing multiple servers.

This is where SSH key authentication comes in.

Instead of relying on a password for every login, you can generate an SSH key pair on your Ubuntu machine and configure your remote server to recognize it. Once everything is set up correctly, you can connect without entering your server account password each time.

In this guide, we’ll walk through how to generate SSH keys on Ubuntu 24.04, copy your public key to a remote server, configure key-based authentication, and troubleshoot common problems.

What Are SSH Keys?

SSH keys are a pair of cryptographic keys used to authenticate connections between two computers.

An SSH key pair consists of:

  • Private key: Stored on your local computer. It should never be shared with anyone.
  • Public key: Copied to the remote server you want to access.

When you attempt to connect to a server, the SSH client uses your private key to prove your identity. The server checks whether the corresponding public key is authorized to access the account.

If the keys match and the server accepts the authentication request, the connection is established.

One advantage of this approach is that your private key never needs to be uploaded to the server.

For most modern Ubuntu installations, Ed25519 is a good choice because it provides strong security with relatively small keys and efficient operations.

Prerequisites

Before generating SSH keys, make sure you have:

  • An Ubuntu 24.04 machine or another Linux system with an SSH client.
  • Access to a remote Ubuntu server.
  • A valid username on the remote server.
  • The server’s public IP address or hostname.
  • OpenSSH installed on both machines where required.

If you haven’t configured SSH access on your server yet, check our guide on how to install OpenSSH Server on Ubuntu 24.04.

You should also be familiar with connecting to a remote machine using SSH. Our tutorial on how to connect to an Ubuntu server using SSH covers the basic connection process.

Step 1: Check Whether OpenSSH Client Is Installed

First, open a terminal on your local Ubuntu machine.

Check whether the SSH client is available:

ssh -V

If OpenSSH is installed, you’ll see output showing the installed version.

If the command is unavailable, install the OpenSSH client:

sudo apt update
sudo apt install openssh-client -y

You can now use SSH commands from your Ubuntu machine.

Step 2: Generate SSH Keys on Ubuntu

The next step is generating a new SSH key pair.

Run the following command:

ssh-keygen -t ed25519 -C "[email protected]"

Replace [email protected] with your email address or another descriptive identifier.

The -t ed25519 option specifies the key algorithm, while -C adds a comment to help identify the key later.

After running the command, you’ll see a prompt similar to this:

Generating public/private ed25519 key pair.
Enter file in which to save the key
(/home/username/.ssh/id_ed25519):

Press Enter to use the default location.

Next, SSH will ask you to enter a passphrase:

Enter passphrase (empty for no passphrase):
Enter same passphrase again:

I recommend setting a strong passphrase to protect your private key if someone gains access to your local files.

Once the process finishes, you should see confirmation that both keys have been saved.

Your SSH key pair is now ready.

Where Are SSH Keys Stored?

By default, OpenSSH stores user authentication keys inside the .ssh directory:

~/.ssh/
├── id_ed25519
└── id_ed25519.pub

These files serve different purposes:

FilePurpose
id_ed25519Private key
id_ed25519.pubPublic key

Keep the private key secure. The public key can be shared with servers where you want to authenticate.

Important: Never send your private key (id_ed25519) to anyone or upload it to your remote server.

Step 3: Verify Your SSH Key Pair

Before copying the key to your server, it’s a good idea to verify that both files exist.

Run:

ls -l ~/.ssh/

You should see your id_ed25519 and id_ed25519.pub files.

You can also display your public key:

cat ~/.ssh/id_ed25519.pub

The output should look similar to this:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA... [email protected]

The actual key will be much longer.

You can safely copy this public key when configuring authentication on your remote server.

Step 4: Copy Your Public Key to the Remote Server

Now that you’ve generated your SSH key pair, you need to add the public key to the remote server.

The easiest way is using ssh-copy-id.

Run this command from your local Ubuntu machine:

ssh-copy-id username@SERVER_IP

Replace the values with your actual server username and IP address.

For example:

ssh-copy-id [email protected]

If your server uses a custom SSH port, specify it with the -p option:

ssh-copy-id -p 2222 [email protected]

You’ll normally be asked for your remote account password during this process.

Once authentication succeeds, ssh-copy-id copies your public key into the remote user’s ~/.ssh/authorized_keys file.

You should receive a confirmation that the key has been added.

That’s it. Your remote server now recognizes your public key.

Step 5: Test SSH Key Authentication

Before changing any SSH server settings, test your connection.

Run:

ssh username@SERVER_IP

For example:

ssh [email protected]

If everything is configured correctly, the server should allow you to log in using your SSH key.

If you protected your private key with a passphrase, you may still be asked to enter that passphrase. This is normal.

Remember that an SSH key passphrase is different from your remote server account password.

If the connection works without asking for your account password, public key authentication is working.

Step 6: Use SSH Keys With a Custom Port

Some administrators change the default SSH port from 22 to another port.

If your remote server uses a custom port, you can connect using:

ssh -p 2222 username@SERVER_IP

For example:

ssh -p 2222 [email protected]

You can also specify a particular private key:

ssh -i ~/.ssh/id_ed25519 -p 2222 [email protected]

The -i option tells SSH which private key file to use.

This is particularly useful when you manage multiple servers with different SSH keys.

Step 7: Configure SSH Client for Easier Connections

Typing the username, IP address, port, and private key location every time can become repetitive.

Fortunately, OpenSSH supports client-side configuration through the ~/.ssh/config file.

Create or edit the file:

nano ~/.ssh/config

Add the following configuration:

Host myserver
    HostName 192.168.1.100
    User ubuntu
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Save the file and set the appropriate permissions:

chmod 600 ~/.ssh/config

Now you can connect to your server using a much shorter command:

ssh myserver

SSH will automatically use the hostname, username, port, and private key specified in the configuration.

If you manage several VPS instances, you can add multiple host entries to the same file.

For example:

Host production
    HostName 192.168.1.100
    User ubuntu
    Port 22
    IdentityFile ~/.ssh/id_ed25519

Host staging
    HostName 192.168.1.101
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_staging

You can then connect using:

ssh production

Or:

ssh staging

This makes everyday server administration much easier.

Step 8: Fix Common SSH Key Authentication Problems

Sometimes SSH key authentication doesn’t work on the first attempt.

In many cases, the problem is related to file permissions, an incorrect key path, or SSH server configuration.

Here are a few things worth checking.

1. Incorrect SSH Directory Permissions

OpenSSH checks file permissions before accepting authentication keys.

On your remote server, check the permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod go-w ~/

These commands ensure that the SSH directory and authorized keys file have appropriate permissions.

2. SSH Keeps Asking for the Server Password

If SSH continues asking for your account password, check whether the public key was copied correctly.

On the remote server:

cat ~/.ssh/authorized_keys

Make sure your public key appears in the file.

You should also verify that public key authentication is enabled in the SSH server configuration:

sudo nano /etc/ssh/sshd_config

Check the following setting:

PubkeyAuthentication yes

If you make changes, validate the configuration:

sudo sshd -t

If validation succeeds, reload the SSH service:

sudo systemctl reload ssh

Keep your existing SSH session open while troubleshooting. This helps prevent accidental lockout.

3. Permission Denied (publickey)

If you receive an error like:

Permission denied (publickey).

Try connecting with verbose output:

ssh -v username@SERVER_IP

For more detailed information, use:

ssh -vvv username@SERVER_IP

This helps identify which key the SSH client is offering and why authentication may be failing.

You can also check the SSH service logs on Ubuntu:

sudo journalctl -u ssh -n 50 --no-pager

Look for authentication errors or permission-related messages.

4. SSH Is Using the Wrong Private Key

If you have several SSH keys on your local machine, the client might attempt to use a different key.

Specify the correct private key explicitly:

ssh -i ~/.ssh/id_ed25519 username@SERVER_IP

You can also use the IdentityFile setting in your SSH client configuration.

Step 9: Manage Your SSH Key Passphrase With ssh-agent

Entering your private key passphrase every time you connect can be inconvenient.

You can use ssh-agent to temporarily manage unlocked private keys.

Start the SSH agent:

eval "$(ssh-agent -s)"

Add your private key:

ssh-add ~/.ssh/id_ed25519

Enter your passphrase when prompted.

After that, the SSH agent can use the loaded key for subsequent connections during the agent’s lifetime.

To see which keys are currently loaded:

ssh-add -l

To remove all identities from the agent:

ssh-add -D

This is especially useful when working with multiple servers throughout the day.

A Small Real-World Administration Scenario

Imagine you’re managing several Ubuntu VPS instances for different environments: development, staging, and production.

Initially, you connect to each server by entering its username, IP address, and account password. It works, but switching between servers repeatedly can become frustrating.

There’s also the risk of using inconsistent passwords across environments or storing credentials in places where they shouldn’t be.

A more practical approach is to generate an Ed25519 SSH key pair, install the public key on each authorized server, and use an SSH client configuration file to organize your connections.

For example, instead of remembering several connection commands, you can simply run:

ssh production

Or:

ssh staging

The SSH client handles the connection details automatically.

This small configuration change can make routine server administration more efficient, particularly when you’re responsible for multiple Linux environments.

It’s also a good reminder that secure server access isn’t just about choosing a strong password. Managing authentication methods properly is an important part of maintaining a reliable infrastructure.

Security Recommendations When Using SSH Keys

SSH key authentication is a useful security improvement, but it still requires proper key management.

Keep these recommendations in mind:

  • Protect your private key: Never share or upload your private key to a remote server.
  • Use a passphrase: Protect your private key with a strong passphrase.
  • Prefer Ed25519: Use modern key algorithms such as Ed25519 for new SSH key pairs.
  • Use separate keys when appropriate: Consider separate keys for different environments or services.
  • Remove unused keys: Review authorized_keys periodically and remove keys that are no longer needed.
  • Keep a backup: Store an encrypted backup of important private keys in a secure location.
  • Test before disabling password authentication: Make sure key-based access works and that you have an alternative recovery method before changing server authentication settings.

Once you’ve confirmed that key-based access works, you can consider disabling password authentication to reduce exposure to password-based login attempts. Do this carefully, especially on remote VPS instances where losing SSH access could require provider-level console recovery.

Conclusion

Generating and using SSH keys on Ubuntu is a straightforward process that can make remote server access both more convenient and secure.

With OpenSSH and Ed25519, you can create a key pair, transfer the public key to your remote server, and authenticate without repeatedly entering your server account password.

For administrators managing multiple Ubuntu servers, combining SSH keys with a client configuration file and ssh-agent can make daily operations considerably easier.

If you’re setting up a new Ubuntu VPS, SSH key authentication is one of the first improvements worth considering.

Frequently Asked Questions

1. How do I generate an SSH key in Ubuntu 24.04?

Run ssh-keygen -t ed25519 -C "[email protected]" in your terminal. Follow the prompts to choose a key location and set a passphrase. Your private and public keys will normally be saved in ~/.ssh/.

2. Where are SSH keys stored in Ubuntu?

SSH user authentication keys are stored in the ~/.ssh/ directory by default. The private key is usually named id_ed25519, while the corresponding public key is id_ed25519.pub.

3. Should I share my SSH public key or private key?

You can share your public key with trusted servers and services that need to authenticate you. Never share your private key because it is used to prove your identity.

4. Is Ed25519 better than RSA for SSH authentication?

Ed25519 is generally a good default for new SSH keys because it provides strong security, smaller key sizes, and efficient performance. RSA with a 4096-bit key remains an alternative when compatibility requirements call for it.

5. Can I use the same SSH key for multiple servers?

Yes. You can install the same public key on multiple servers and use the corresponding private key to access them. However, using separate keys for different environments can make access management and key revocation easier.

6. Why does SSH still ask for a password after adding my public key?

Common causes include incorrect file permissions, an incorrectly copied public key, the wrong username, an incorrect private key path, or public key authentication being disabled on the server. Use ssh -v username@SERVER_IP to investigate the connection.

7. Do I need to enter my SSH key passphrase every time?

Not necessarily. You can use ssh-agent and ssh-add to keep your unlocked private key available for subsequent SSH connections during the agent’s lifetime.

8. Can I disable password authentication after setting up SSH keys?

Yes, but only after testing key-based access and confirming you have a recovery method. Disabling password authentication before verifying your SSH key can lock you out of your server.