14 SSH Key Management Best Practices You Need to Know
3 votes, average: 4.67 out of 53 votes, average: 4.67 out of 53 votes, average: 4.67 out of 53 votes, average: 4.67 out of 53 votes, average: 4.67 out of 5 (3 votes, average: 4.67 out of 5, rated)

14 SSH Key Management Best Practices You Need to Know

How well you manage and secure your secure shell key lifecycle in part determines the security of your network and other IT environments. Here are several SSH key management best practices that will help you get started

SSH key management is an oft-overlooked element of identity and access management (IAM). Here at Hashed Out, we typically focus on public key infrastructure (PKI)-based forms of authentication and digital identity because that’s our specialty and area of expertise. However, we have to acknowledge that secure shell (SSH), which is something virtually all companies use in some capacity, also needs some love in terms of coverage every once in a while.

This is why we’ve decided to tackle an article that addresses some of the most important SSH key management best practices for your business. But what are the top 14 SSH key management best practices you need to implement now and how do they benefit your organization?

Let’s hash it out.

What Is SSH Key Management? The Role of SSH Keys in Identity & Access Management

Before we jump right into defining SSH key management, let’s first quickly rehash what secure shell is, what SSH keys are, and how organizations use them for user-friendly authentication and increased data security.

What Is SSH and What Does It Do?

SSH, which stands for secure shell, is a cryptographic network protocol that allows secure authentication and data communications between two devices via open channels. SSH is critical for network and general system administration (such as for managing firewalls, networks, servers, etc.). As something that’s literally built into Unix and Linux servers, it’s a client-server model system that aims to secure remote access between users and critical systems via insecure connections.

More simply stated, SSH is the numero uno system that IT admins use to log in to servers and other Linux machines to manage them remotely. The way it works is when authorized users log in to a secure shell environment to authenticate, their devices gain access to one or more devices or resources within your secure IT environment. As such, it’s easy to see how SSH authentication plays a critical role in your organization’s identity and access management processes.

Screenshot of an SSH access management server root admin access login prompt
A screenshot by Jeremy Caban, The SSL Store’s IT administrator and DevOps engineer.

When you use SSH to log in using admin root access, you have complete control over the machine. SSH access serves as your key to your kingdom, giving you control of all commands and files. So, as you can imagine, protecting that kind of privileged access through proper SSH key management is crucial to your organization’s security. (Going out a limb here to say that SSH security is important to you since you’re still here and have read this far, at least…) 

SSH access management screenshot showing root access
A screenshot by Jeremy Caban, TheSSLstore.com’s IT administrator and DevOps engineer.

SSH Authentication (and Where SSH Keys Fit Into the Picture)

SSH authentication can occur in a few different ways. However, the two we’re going to focus on today are password- and SSH-key-based authentication methods. And our primary focus, as your can probably guess from this article’s title, is going to focus on SSH key management best practices specifically.

  • Password-based SSH authentication occurs when a user manually enters their login credentials to access a secure resource. An example of this is when one of your coworkers enters their individual username and password into a login field to access a file in, say, Amazon Web Service’s Elastic Computing Cloud (AWS EC2).
  • SSH-key-based authentication allows a user to authenticate automatically using a cryptographic keypair. These keypairs, which consist of private and public keys, are used to authenticate users (their devices) and hosts. Basically, you use the local machine that has the private key to authenticate to AWS once. After that, all SSH interactions between the two devices will rely on the key pair for authentication.

In a nutshell, SSH keys are another form of machine identity and authentication. They enable your organization’s authorized, authenticated users to access critical systems to perform their jobs. But SSH keys, unlike traditional PKI keys that we use with X.509 digital certificates, don’t have public keys that are signed by a certificate authority (i.e. certification authority or CA) like DigiCert or Sectigo. (In SSL, you generate your SSL keys yourself through the CSR process, and the public key is what the CA signs as part of your certificate.) And even though SSH authentication involves a process that’s similar to the SSL/TLS handshake, its authentication process isn’t as complex.

Related: Secure Your Domain & Sub-Domains with a RapidSSL Wildcard Certificate

Here’s a quick (and simplified) overview of how SSH authentication works:

A simplified example of how SSH authentication works that's illustrated as a conversation between a user and the SSH server they're connecting to
This graphic is a simplified view that breaks down how the SSH authentication process works using SSH public and private keys.

But when comparing password- and key-based SSH authentication methods, which option is better or more secure? EdgeScan’s 2021 Vulnerability Statistics Report shows that some of the biggest growths in terms of exposure for organizations in 2020 relate to the use of insecure SSH credentials.

“Remote desktop (RDP) and Secure Shell (SSH) exposure increased by around 40%, likely due to the increase in remote working due to covid-19. RDP (and similar services) are easy and commonly used avenues for brute force or credential stuffing attacks, against weak user credentials.”

EdgeScan’s report shows that in their sample of one million public-facing endpoints’ exposed services, SSH represented 3.8% (38,000) in terms of “remote system login and management.”

When properly managed, SSH keys offer an authentication method that’s more secure than using traditional login credentials alone. Why? Because keys are resistant to brute force attacks. But the security and effectiveness of those keys depend on how well your organization keeps track of them. This is where following SSH key management best practices comes into play.

SSH Key Management — It’s How You Control the Keys to Your Kingdom

SSH key management is the combination of policies, processes and tools that enable you to protect and manage those digital key pairs. Secure shell keys allow users to authenticate themselves to your network, servers, or other systems and securely share files without continually logging in using a username and password.

Some of the benefits of effective SSH key management include:

  • Having full visibility into your access keys at all times (this helps you to ensure each is protected from theft).
  • Knowing that you’re not accidentally re-using keys across systems and users (yeah, this can happen).
  • Ensuring you don’t get locked out of a server or system if you lose an access key (accidents happen).
  • Having the ability to immediately change or revoke access for employees (for example, when someone leaves the company).
  • Being able to change or revoke access if a key becomes compromised (you’ve gotta act quickly in this situation).

Much like PKI key management, successful SSH key management revolves around how well you can protect and keep track of your organization’s public and private keys. This means using effective methods to generate, store, rotate, revoke, and use them in ways that keep them out of the hands of cybercriminals and other unauthorized users. This requires processes that ensure proper SSH key provisioning, terminations, and monitoring across all of your IT environments.

And this can be tricky, considering that without proper access management and approval processes in place, users can simply issue themselves access privileged access to your most critical systems. This is why having a strong SSH key management program in place is essential. It’s also why SSH key management should be part of your access and IT risk management and remediation processes, policies and strategies.

SSH Key Mismanagement Leaves You Vulnerable and Non-Compliant with Industry Regulations

You should understand by now that mismanaging your SSH keys leaves your organization at risk of credential compromise, data theft, and data breaches, but did you know it also makes you non-compliant with some industry regulations? Many well-known regulations — HIPAA, GDPR, PCI DSS — all require data security and privacy protections. Some of these requirements relate to managing access to that data, which implies or specifies the use of cryptographic keys such as SSH keys.

Health Insurance Portability and Accountability Act (HIPAA)

HIPAA’s Security Rule requires covered organizations to implement role-based access measures to protect electronic protected health information (ePHI) data when it’s in use, in transit, and at rest. This is something that SSH key management plays a role in.

Payment Card Industry Data Security Standards (PCI DSS)

PCI DSS Requirement 4 specifies that covered organizations must use encrypted transmissions to transmit cardholder data over open networks. This requires the use of cryptographic processes and components like public-private keypairs.  

PCI DSS Testing procedure 2.3 requires the use of “strong cryptography” for all non-console administrative access. According to the PCI DSS guidance specified in 2.3c: “To be considered ‘strong cryptography,’ industry recognized protocols with appropriate key strengths and key management should be in place as applicable for the type of technology in use[.]”

General Data Protection Regulation (GDPR)

The European Union’s GDPR requires the secure processing of covered personal data. GDPR Article 32 specifies the use of “appropriate technical and organisational measures,” which include data encryption and secure access to that data. (This includes the use of public key encryption methods to secure access.)

Needless to say, if bad guys manage to get their hands on your employees’ SSH keys, it can lead to many dire and costly consequences. This is why SSH key management should be a priority for every business regardless of size.

14 SSH Key Management Best Practices You Need to Implement Now

Research from Acunetix’s 2020 Web Application Vulnerability Report shows that 15.5% of the 5,000 scan targets they analyzed had SSH-related vulnerabilities. A large part of this is due to SSH key mismanagement. That’s because the security of SSH keys is only as good as the SSH key management and auditing practices you implement.

A Giphy graphic that shows a locked door isn't secure because it swings both ways. Graphic source: https://giphy.com/gifs/door-bathroom-lock-Fz6gBKK52Nfyg
This Giphy graphic illustrates how strong and effective 2048-bit SSH keys are when they’re not managed correctly…

Effectively managing your SSH keys entails knowing:

  • The precise number of SSH keys that exist within your IT environment,
  • When and how each key is used and which system(s) it has access to,
  • How old each key is and when you need to replace it, and
  • How to safely generate, store, revoke, and remove the keys within your network and overall IT environment.

The National Institute of Science and Technology (NIST) provides in-depth guidance for SSH access management in their interagency report 7966 (NISTIR 7966). They cover many vulnerabilities, including:

  • SSH implementation issues,
  • Access control configuration issues,
  • Unintended SSH key usage, and
  • Unknown or unaudited active keys (and the risks associated with them).

However, there’s a lot of material to cover there and, frankly, we don’t think you’re here to get into the nitty-gritty of all of that now. That’s why we’re going to take a bit more of a high-level approach to SSH key management in this article.

So, without further ado, let’s break down the 14 SSH key management best practices you can put to work now to protect and manage access to your organization’s IT environments and data. These best practices will be divided up into four overarching categories that will help make things easy to follow.

Use Tools That Increase Visibility of the SSH Keys Within Your Network & IT Environment

SSH key visibility is the first best practice we’re going to mention because you can’t protect or secure your organization’s SSH keys if you don’t know that they exist. It would be like being in charge of the U.S. Secret Service and having to protect the President without knowing in advance where they’re traveling to and whom they’re meeting with.

Likewise, effective key governance requires you to have a clear picture of what’s going on within your network and IT environment. Part of this entails maintaining a current (ideally centralized) inventory of all of your SSH keys and how they’re used across your servers, hosts, and other IT infrastructure. This will help you to identify:

  • Duplicate SSH keys or access,
  • Identify “orphaned” or “rogue” SSH keys that are still active and valid, and
  • Who is using what keys to access which system(s).

If you don’t know what keys are used by which systems or users, then you’re in for a world of pain. That’s because bad guys like hackers and other cybercriminals might figure out which keys have access to specific systems before you do and exploit them.

So, how can you figure out where all of these different keypair components live and what they have access to?

1. Use an SSH Key Manager to Discover SSH Keys and Enable Automation

Using a reliable SSH key management tool is an easy way to manage the key management lifecycle within your organization. An SSH key manager helps you discover wherever any keys exist within your IT infrastructure and what they control or have access to. This helps you identify potential vulnerabilities that bad guys can exploit via temporarily forgotten, orphaned or rogue SSH keys.

In their eBook “6 Steps to Take Back Control of Your SSH Keys,” Keyfactor describes orphaned keys as authorized public keys whose private keys’ locations aren’t known. Forgotten keys are the temporary-use keys admins create for specific tasks but forget to remove later on. As the name implies, rogue keys are unauthorized tools that may be created out-of-band.

SSH key management tools allow you to manually manage your keys while also enabling you to automate many of these functions. They can also help you to avoid or alleviate key sprawl issues, which is particularly useful for large organizations and corporations that have to manage thousands or even millions of SSH keys (in the case of Fortune 500 companies) within their environments.

However, it’s important to note that not all key management platforms support SSH keys. Some examples of SSH key management platforms that do allow you to protect and manage them are:

  • AppViewX,
  • Keyfactor’s SSH Key Manager,
  • ManageEngine’s Key Manager Plus, and
  • Venafi’s SSH Protect.

Use Secure Methods to Store, Backup & Permit Authorized Access to Keys

A graphic illustrating a checklist and set of processes to follow

As I mentioned earlier, SSH keys come in key pairs that consist of private and public keys. The private key is the one that you store on the user’s device who will use the key for access. For example, their office laptop or smartphone. The public key, on the other hand, is what you’ll need to store on the machine that users will connect to (such as a server). This could be your network, a server, or a variety of other systems.

Every SSH private key should be protected with a unique and hard-to-guess passphrase. What this does it add another layer of security against brute force attacks and tools. This way, even if a bad guy manages to gain access to your private key, they can’t do anything with it because they can’t guess the password. Every time you replace this key, you should also use a new, strong passphrase.

The private key should also remain on the device that you generated it on. The only exception would be if you decide to store it in a secure key vault or hardware security module (HSM). Just be sure to never, ever share an SSH private key over a network!

Effective SSH key management requires secure backups and storage of these keys. Part of this means knowing how and where to save keys so that they don’t get leaked or become compromised in another way. For example, you never want to store your SSH key information in whatever folder of code you’re sharing around because they could wind up getting leaked.

2. Use a Key Vault and Physical SSH Key Storage

One option is to use a trusted and reputable key vault (such as Azure Key Vault or AWS Key Management Service). This is a secure, centralized place where you can store many types of cryptographic elements securely (certificates, PKI keys, SSH keys, and other secrets) so that authorized web applications and entities still have access to it as necessary.

Another option for secure key storage is to store your SSH keys in a physically secure offline environment. This should mean storing the secure device that holds the keys under lock and key and in a secure area that only select authorized individuals have access to.

3. Implement Key Escrow to Permit Access to SSH Keys by Only Authorized Entities

A great way to ensure your private keys stay secure while still being accessible is through the use of a process known as key escrow. This basically means using a trusted third party or tool to hold on to a copy of your SSH private key. An advantage of using a key escrow is that it serves as a backup should something unforeseen happen to the original key.

An example of using key escrow is when an IT admin uses a third-party tool to store the keys so authorized entities that need them can get access to them. To set this up, they have to determine where and how to save the key and who gets access to it. Some key escrow capabilities may already exist within your system (for example, it may be part of your Active Directory).

In some ways, the concept of key escrow is similar to how your mortgage company manages an escrow account for you. They collect a set amount of money from you each month, part of which they store in an escrow account. This account then is what they use to pay your annual property taxes and homeowner’s insurance when they come due each year.

Implement and Follow Effective Access Management Best Practices

It’s no surprise that effective user access management is also an SSH key management best practice. After all, SSH keys are just another method of authentication that you can use in lieu of traditional credentials (usernames and passwords) for your users. So, this means that your SSH key management processes and policies should align with your access controls.

As we touched on earlier, NIST also provides guidance regarding SSH user access and key management. This can be found in their interagency report 7966 (NIST IR 7966) that we referenced earlier. It also affects the control families they address in special publication 800-53 (SP 800-53).

4. Implement and Enforce Strict SSH Key Management Policies and Processes

SSH key management is only as good as the policies you have in place and enforce. If you have policies on paper that are never followed or applied, they’re essentially worthless. These policies should be the backbone that governs your SSH key infrastructure and processes, and they should also ensure accountability by those who are responsible for them.

Suppose you have policies in place that clearly outline everyone’s responsibilities and roles, and those policies are regularly enforced. In that case, carrying out those functions is less likely to get pushed to the back burner of priorities and become forgotten. 

5. Create, Manage and Document Individual User Accounts (Avoid Creating Shared Account Credentials)

While it may be easier to create a few shared accounts, more effective credential management dictates that it’s best to create an individual account for every employee. This means creating that you’ll need to create SSH key pairs for everyone who needs access to specific systems.

Part of this process should include formal access approvals and accompanying documentation. This helps to provide a record about why each key’s access is necessary and which key(s) a user’s access is tied to. While all of this is useful for SSH key management in general, it’s especially useful by helping you keep your ducks in a row when terminating access for users.

6. Implement a Policy of Least Privilege (POLP)

The thought here is that only authorized users should have privileged access. To ensure this, you can configure an individual’s SSH key so that it’s associated with an account that limits access to what that user needs.

It’s also best to remove old access whenever employees change jobs within your company or leave your company altogether. This brings us to our next talking point…

7. Remove Old SSH Public Keys (Mitigate Key-Based Risks)

Orphaned and forgotten SSH keys are a big issue for businesses. These unaudited keys are essentially forgotten active keys that create backdoors that cybercriminals or even disgruntled former employees can exploit. This is a particularly important step since you don’t want valid SSH keys out there that can access your systems that you don’t know about.

Whenever someone leaves your company, you should have policies and processes in place for how to terminate access for any accounts associated with that user. One such access management method is to remove all public keys that are associated with them. This will prevent their private keys from allowing access to authorized systems. Otherwise, you’re left with keys that bad guys can try to exploit.

One of the ways to harden your SSH security is to ensure that everything is properly configured — that you’re dotting your is and crossing your Ts. You can set your SSH configurations using various tools, terminals, and applications such as PowerShell or MacOS’s terminal.

Here are a few SSH key management items you should double-check to make sure you’re doing right:

SSH keys are versatile in that they can be generated in multiple size options. However, NIST IR 7966 specifies that “user keys must conform to organizational standards for minimum key lengths used with approved algorithms.”

This is why many organizations nowadays opt to generate RSA 2048 SSH keys. The idea here is that a key of this size helps to make factoring attempts too impractical (and costly) for attackers to carry out. So, to ensure that all SSH key generations within your organization meet the minimum, you must specify a minimum key size.

You can generate your keypair in a couple of ways. One method is by generating it directly on the server itself with the command ssh-keygen -t rsa and following the prompts, which includes creating a passphrase for the private key. I’m going to borrow a screenshot from one of my former colleague’s articles that shows how this looks:

A screenshot demonstrating the SSH key generation process
Image source: Our article “Secure Shell: What Is SSH?

Another method is to use standalone software (such as PuTTYGen) that allows you to create your public-private keypairs.  

9. Invalidate Your SSH Keys After a Certain Period

What’s interesting about SSH keys is that, unlike PKI certificates, they don’t technically “expire” after any given period. Sure, they can be revoked, but in order for them to so-call “expire” in the sense of no longer being usable, you have to manually force your server to reject old keys or delete them outright from your IT systems. (Hmm, maybe I should say “retire” instead of “expire” here.)

But regardless of the choice of verbiage, why is invalidating SSH keys necessary? You don’t want to leave SSH keys valid indefinitely because doing so poses significant security risks. For one, improperly managing keys results in active keys that offer access to critical systems never being removed. Another huge concern is employee turnover. If employees who leave the company have access because their keys are still active, it’s a huge (and entirely avoidable) liability.

10. Rotate Keys Regularly

Because SSH keys don’t expire, a common practice is to rotate your keys regularly. Although you should have processes in place to ensure that inactive keys are removed, we get that life happens and some things don’t always happen as they should. So, regular key rotation helps to prevent compromised keys from being exploited by bad guys.

However, NIST IR 7966 does note that key rotation can negatively affect or break external keys. These keys are those that either come from outside organizations or are internal keys that authenticate and access systems external to your environment. Proper SSH key rotation means:

  • Replacing existing identity keys with new authorized keys that you generate, and
  • Updating all corresponding systems to reflect those key changes.

But how regularly should you rotate keys? Eh, that answer depends on who you ask. For example, NIST IR 7966 is pretty vague about that and just hedges their bets by saying that keys should be rotated “regularly.” But in Appendix F, they include a link to a 2013 NIST document draft that specifies the following:

“Authentication credentials for all trust relationships leading to moderate-impact and high-impact systems MUST be rotated every 12 months, and it is RECOMMENDED that trust relationships leading to low-impact systems be rotated every 12 months. It is recommended that all keys be rotated as part of a remediation process to ensure that any previously leaked keys cease to be usable.”

Trend Micro, on the other hand, is more specific. They say that you should rotate SSH public keys approximately every month-and-a-half (i.e., every 45 days).

Toby Gaff, Director of Solutions Engineering at Keyfactor, cautions randomly rotating keys without having a plan:

“But here’s the challenge when it comes to SSH Key, most of them are not used for interactive purposes. We could see two different requirements existing for key rotation. One based on application/system keys and the other on interactive/user keys. We believe that keys should be rotated more frequently for interactive/user keys (kind of like passwords), but having such a policy is kind of pointless unless you can report on it or enforce it.”

This brings me to my next point about changing keys…

11. Use Different SSH Keys For Different Users, Hosts, and Environments

A good rule of thumb is to use unique SSH keys for different users and environments. For example, the SSH keys you use for admins in production shouldn’t be the same as the ones you use for accounts in development. This way, if someone manages to compromise an SSH key for a user in one environment, they can’t turn around and use that same key to access other environments across your systems.

12. Change the Default SSH Port and Disable Port Forwarding

SSH uses TCP port 22 by default, but that doesn’t mean you’re automatically stuck using it. Jeremy Caban, IT administrator/DevOps Engineer at TheSSLstore.com, weighs in on this best practice:

“One thing that comes to mind is the school of thought around using a different port for SSH. With proper security measures in place, it makes this pretty much a non-issue for the most part. But still, it never hurts to have more security layers in place.”

While changing the communication port isn’t necessarily a requirement, it’s a great best practice because it helps to reduce automated attacks (such as brute force attacks). And considering that there are 65,536 TCP communication ports you could choose from, it’s only right that you make it harder for threat actors to try to figure out which port you’re using. (Make ‘em work for it, right?)

According to the Internet Assigned Numbers Authority (IANA), port numbers are typically broken down into three sets of ranges (some of which may require IANA registration):

  • System ports (0-1023),
  • User ports (1024-49151), and
  • Dynamic/private ports (49152-65535). Note: these ports don’t require IANA assignment.

You can change your client program’s port access specifications manually in your SSH configuration files via the Linux command line. This forces SSH to connect via the port number of your choice. You can also configure your firewall to block access to port 22 altogether or restrict access to it from trusted hosts as well. For specific directions on how to change your default SSH port, check out this article Caban recommends by linuxhint.com.

SSH.com also recommends disabling port forwarding if you don’t expressly need them. Port forwarding refers to the process of “tunneling application ports from the client machine to the server […] or vice versa.” While it has some legitimate uses for organizations, it’s also something that cybercriminals can exploit as a backdoor into your internal network.

13. Avoid Using Weak Versions of SSH Protocols

Something that can easily get overlooked is which SSH protocols you use for clients and servers. For example, you want to avoid using SSH version 1 (SSH-1) and should set it to use SSH version 2 (SSH-2) by default instead. A key reason why you want to use the latter over the former is that it’s more secure and isn’t vulnerable to some of the same security exploits as SSH-1.

14. Keep Your SSH Servers and Clients Patched and Up to Date

One of the most overlooked areas of cybersecurity relates to applying patches and other updates. Manufacturers issue updates and patches to address bugs and critical vulnerabilities, which may leave your organization and data open to attack.

Frankly, patch management is an area where you can use automation easily, yet many companies still manage to fall behind on implementing these manufacturer-issued fixes.

Final Thoughts on Effective SSH Key Management

Much like SSL and TLS, SSH itself is a cryptographic network protocol that enables both secure authentication and data communications via open channels. SSH keys, which serve as an alternate means of user access, allow users to authenticate themselves to servers in a more secure way than using traditional login credentials alone. However, for this connection and authentication process to be used securely, it requires stringent key management policies, processes, and implementations.

To ensure the security of your organization’s secure shell usage, you need to have visibility and know what authorization and access every SSH key has. My hope is that this article serves as a useful introduction and a resource that helps you implement better and more secure SSH key management best practices.  

As always, if you have any additional suggestions or best practices you’d like to mention, feel free to share your thoughts in the comments below.


Casey Crane

Casey Crane is a regular contributor to and managing editor of Hashed Out. She has more than 15 years of experience in journalism and writing, including crime analysis and IT security. Casey also serves as the Content Manager at The SSL Store.