Certificate expiration can happen to anyone, let this be a cautionary tale.
Anyone running an inter-office Cisco-powered Virtual Private Network (VPN) is probably going to need to install an update to ensure everything continues working properly. That’s because Cisco inadvertently allowed an SSL certficate embedded in Switchzilla’s Application Policy Infrastructure Controller Enterprise Module (APIC-EM), a software-defined networking controller, to expire on July 13th.
In a Field Notice issued yesterday, August 6th, Cisco notified its users of the issue:
The APIC-EM Public Key Infrastructure (PKI) broker fails in affected software versions. As a result, the APIC-EM instance becomes unable to provision trustpoints. APIC-EM instances with this problem are not able to generate new device Secure Sockets Layer (SSL) certificates or use the APIC-EM Intelligent WAN (IWAN) application to deploy new hub/branch sites.
The cause of this was the expiration of the embedded SSL certificate, thus preventing the creation of any new trust points.
Here’s what that means in layman’s terms. The embedded SSL certificate was effectively serving as a root for an open-source certificate authority called EJBCA. When the root certificate was valid, it could issue, renew and revoke the X.509 certificates being used for authentication and encryption across the VPN.
When the root goes bad, or expires, it means that you can no longer issue new certificates and it threatens to render all existing certificates that chain back to the expired one to become untrusted. That latter portion doesn’t appear to have actually occurred, rather Cisco VPN users are just being blocked from creating new end points as a result of not being able to issue the proper digital certificates.
As per Cisco, a fix is on its way.
A fix for this problem will be available in APIC-EM Release 1.6.3. Alternatively, a qualified Cisco engineer can apply a manual patch to affected systems. Contact the Technical Assistance Center (TAC) for assistance with the manual patch.
Now, the point of this discussion isn’t criticize Cisco, but rather to point out just how much of an effect a single expired SSL certificate can have. We harp all the time on not letting certificates expire without having a replacement ready and this is why. Just one single certificate has the ability to knock out a large portion of your defensive posture.
That’s why, especially at the Enterprise level, certificate visibility and having the proper tools to manage certificate life cycles is so criticially important. Speaking with some of our enterprise account managers, the biggest challenge facing most organizatons isn’t procuring the certificates – that’s never been easier – it’s managing them.
Does your organization have the tools in place to see and manage all of your digital certificates? Would you be interested in learning how?
As always, feel free to leave any comments or questions below.