(No Ratings Yet)

Loading...

• Facebook
• Twitter
• LinkedIn
• Mail

December 8, 2018 Comments closed

Ericsson Outage: Expired Certificate knocks millions of UK mobile phones offline

in Everything Encryption

Tens of millions in the UK were unable to call or text, nor use 4G connections

On Thursday, millions of people in the UK were without cellular coverage following the expiration of a digital certificate associated with Ericsson’s network. Ericsson, which is a Swedish cellular company, manufactures myriad back-end equipment for the world’s cellular networks.

Thanks to an expired digital certificate in a version of Ericsson’s management software that is widely used by European telecommunications companies caused downtime for millions of cellular users in Europe, as well as Japan.

An initial root cause analysis indicates that the main issue was an expired certificate in the software versions installed with these customers. A complete and comprehensive root cause analysis is still in progress. Our focus is now on solving the immediate issues.

This issue caused myriad problems, not only for Ericsson but for its partners, many of whom had services outages and potentially lost money. The outage has inspired no shortage of vitriol, which isn’t exactly going to be a shot in the arm for Ericsson’s reputation.

“I am both shocked and at the same time not at all surprised that certificate expiration was behind all of this service disruption,” writes Davey Winder of Forbes. “Shocked as I would have expected a company as large as Ericsson to know better and have the relevant failsafe processes in place to prevent such an event. It does, after all, describe itself as “one of the leading providers of Information and Communication Technology to service providers” and approximately 40% of the world’s mobile traffic is carried through its networks. I’m not surprised though, because disruptive certificate expiry is something that those of us who inhabit the cybersecurity world are all too familiar with. You probably are as well; how many times has your web browser warned you that a site or service you were about to visit was insecure and so blocked your connection? That will almost always be courtesy of an expired secure sockets layer (SSL) certificate.”

The outages began between 4AM and 5AM and had an impact that extended far beyond just Ericsson’s orbit. It initially affected software used by O2 and its parent company, Telefonica, but eventually the outages showed up downstream on carriers like:

• GiffGaff
• Sky Mobile
• Lyca
• Tesco Mobile

All of whom rely on O2’s network.

Overall 32-million people were without service in the UK and millions more in Asia. O2 technical teams and Ericsson engineers were thrust into an all-nighter, working to restore 4G service until 3:30 AM the next day. Almost 24 hours the collapse.

As of now, both O2’s 3G and 4G services have been fully restored.

From Ericsson’s official statement:

“During the course of December 6, most of the affected customers’ network services have been successfully restored. We are working closely with the remaining customers that are still experiencing issues.”

Ericsson CEO Börje Ekholm has also strangled the culprit personally.

“The faulty software that has caused these issues is being decommissioned and we apologize not only to our customers but also to their customers,” Ekholm said. “We work hard to ensure that our customers can limit the impact and restore their services as soon as possible.”

There is still very little clarity about what kind of digital certificate it was that specifically expired – whether it was a signing certificate that expired and there was an issue with time-stamping or if it was an SSL/TLS certificate – but as we say all the time: this is what happens when your certificate expires.

“This episode illustrates the essential role certificates play in keeping IT infrastructure safe and running, and also the risk that enterprises face if they don’t have a firm handle on the certificates installed in business-critical systems,” said Tim Callan, a senior fellow at Sectigo.

“The proliferation of certificates and ever-increasing complexity of IT infrastructure has made it more and more challenging for IT professionals to stay on top of this component of their networks.”

This is an easily preventable issue, but also an entirely common one. It’s difficult to keep track of certificate expiry when you’re managing a large network. Nobody gets that better than us. But there are plenty of solutions, many of them turn-key, that can help keep you on top of these expiration dates. It’s a small thing, but it can have big ramifications.

“The identity of machines makes the internet run. Machine identities allow our mobile device, networks and computers trust each other. But they expire and networks, allocations and businesses fail,” said Venafi VP Kevin Boeck. “[The] O2 outage is just one more example of how important machine identities are to the economy and when they fail, everything from buses, mobile devices and more, fail. O2’s experience is the same that banks, airlines and the high street have all faced. It’s painful for millions and these problems are only getting worse as we depend more on clouds, mobile devices, AI, and the coming arrival of 5G networks.”

So, once again, sorry for beating a dead horse but stay on top of those expiry dates.

As always, leave any comments or questions below…

• #Certificate Expired