New Study Finds HTTPS Interception is Weakening HTTPS.
A new study has found that HTTPS Interception – the practice of decrypting and scanning HTTPS connections in order to scan traffic for malware and monitoring – is much more prevalent than previously believed. It was also found that devices and software that perform interception significantly reduce the protections of HTTPS and weaken the overall safety of user’s data.
The study was conducted by a large team of academics and professionals including Zakir Durumeric, Zane Ma, Drew Springall, Elie Bursztein, Nick Sullivan (Head of Cryptography at Cloudflare), and Richard Barnes (Firefox’s Security Lead).
Traditionally, an HTTPS connection is made directly between a client (your browser) and a web server. That connection is encrypted and authenticated so that the data sent between the two computers can only be read by those same computers, defeating a wide range of network attacks and monitoring.
“[HTTPS] interception significantly reduces the protections of HTTPS and weakens the overall safety of user’s data.”
But when HTTPS interception is taking place, there is no longer a direct connection between your browser and the server. Instead, a third party, the interception device, stands in the middle. First, an HTTPS connection is made between the web server and the interceptor. This allows the interceptor to decrypt and scan the data once it has received it. The interceptor can then perform traffic inspection and anti-virus scanning. Then another HTTPS connection is made between the interceptor and your browser, and the data is encrypted again.
Two separate and distinct HTTPS connection are needed – the first ‘securely’ delivers the data to the interceptor, and the second ‘securely’ delivers the data to the browser. However, many products that perform interception were found to have poor SSL/TLS capabilities, which compromised the security of the HTTPS connections and left the user with reduced protections.
Due to a combination of outdated software, bad documentation, and the lack of incentive to configure them properly, manufactures and networks who deploy these products are actually weakening their user’s and data’s safety for the benefit of network monitoring and anti-virus scanning.
The Surprising Scale of HTTPS Interception
The study looked at two distinct classes of products. The first are commercial hardware devices known as middleboxes which perform traffic inspection. These are deployed on corporate networks so that administrators can see what is going on while still protecting data as it travels across the public internet. The second is anti-virus software, which is usually run on consumer computers.
The team analyzed nearly 8 billion SSL/TLS handshakes, comparing data of the observed connections to determine when an interceptor was bridging the connection between the server and the client. They found that up to 10.9% of observed connections were intercepted.
This has large implications on the true privacy provided by those HTTPS connections. But the bigger problem is that the interception products are severely weakening HTTPS by using outdated cryptography and failure to implement basic features.
To figure out the scale of the problem, the researchers needed to quantify how much interception was taking place. For observed connections the HTTP User-Agent header (which tells the server what browser and operating system it is talking to) and the parameters of the SSL/TLS handshake were recorded. Comparing that data allowed the researchers to look for inconsistencies that would reveal the connection was being intercepted.
For example, if the User-Agent header said they were talking to a Chrome browser, but the TLS handshake included ciphers that Chrome has never supported, it told them that an interceptor was passing along the browser’s header and making the connection.
Connections were monitored on Firefox’s update servers, on a “set of popular e-commerce sites,” and on Cloudflare’s network, in order to get an accurate sample of internet data.
Cloudflare had the highest rate, with 10.9% of traffic being intercepted. The ecommerce sites had 6.2% of interception, and Firefox update servers had 4%. All of these numbers “are more than an order of magnitude higher than previous estimates,” which estimated the rate was closer to a half percent or less.
The Negative Effects Of Interception
Once they knew how much traffic was being intercepted, the next thing to know was how this was effecting the security benefits of HTTPS. The researchers knew that the interceptor products were using outdated SSL/TLS libraries and often lacked support for modern cryptography and other security features.
To quantify the impact of HTTPS interception, the authors came up with a grading rubric, giving A, B, C, and F grades to the tested products depending on how their SSL/TLS capabilities compared to a modern browser.
Their criteria was very forgiving.
An “A” grade simply meant that the interceptor’s TLS connection was “equivalent to a modern web browser” when it came to encryption and authentication. None of the products they tested had support for advanced HTTPS mechanisms such as HSTS (HTTP Strict Transport Security) and HPKP (HTTP Public Key Pinning), and those factors were left out of the grading. This means that even an “A” grade is still a decrease in overall security because browsers do implement these advanced mechanisms.
A “B” grade was for “suboptimal” connections that were flawed but still cryptographically secure. A “C” grade meant the connection was vulnerable to known attacks; and an “F” was for “severely broken” connections which left out basic functionality like certificate validation altogether, or supported extremely weak encryption ciphers.
Note that the “C” criteria is very forgiving. Most people would consider vulnerability to known attacks to be reason enough to give a failing grade.
Given the low bar, you would think the products performed well. After all, scoring an “A” just means the product needed support for modern cryptography. Browsers do it for free, so commercial products should be able to do just as well, right?
Of the 12 middleboxes only one received an “A.” That was a device from Blue Coat (owned by Symantec). Blue Coat’s ProxySG 6642 properly validated certificates, supported TLS 1.2 and modern ciphers, and mirrored the client’s capabilities. This effectively preserved the security of the client, meaning you were no worse off by using it.
The rest of the middleboxes worsened user security. Six of the middleboxes received a “C” grade and four received an “F.”
“Five of the twelve products introduce severe vulnerabilities that would enable future interception by a man-in-the-middle attacker.” Overall, 62% of observed traffic was made less secure by the middlebox’s poor TLS capabilities, in comparison to the connection being made directly between the browser and server. The authors said the majority of these connections were “severely broken” and questioned the protection they provided.
The upside was relatively small – less than 15% of connections experienced increased security. This would happen when the user’s browser had inferior capabilities compared to the interceptor. Less than a quarter of connections maintained equal security when the middlebox was present.
Anti-virus software performed even worse. The study tested 26 products from 13 vendors including major anti-virus software makers Avast, AVG, and Kaspersky. A whopping 15 products scored an “F” and 9 a “C” (that’s 24 of the 26 products). Only 2 products – both from Avast – scored an “A.”
This means that both consumer and corporate traffic is being severely harmed by their own HTTPS interception products. These products are meant to provide more security – by allowing for network monitoring and anti-virus and malware scanning. While they may very well do that, it comes at the cost of crippling the encryption that protects that data.
In the context of corporate networks, this means that using an interceptor not only makes it easier for you to monitor your traffic, but it’s likely making it easier for other people to monitor your traffic. Poorly encrypted data can be captured and stored by anyone who can monitor the network (such as the government, or your ISP operating under court order from the government) and then decrypted. Some of these connections are weakened so severely that they can be easily decrypted today – while others could be broken in a few years when computing power increases.
The authors pointed out that the problem here is multi-faceted. First, while the internet community has always known HTTPS interception was happening, the scale of it was likely not realized. This means that interception has largely been ignored and there is little consensus if an alternative solution should be sought out.
Then there is of course lots of blame to spread around to middlebox manufacturers and anti-virus vendors. There is no excuse for their horrible SSL/TLS configurations. The problem is bigger than out-of-date libraries and improper implementation; the authors noted that Cisco devices had no documentation on how to configure cipher suites.
But our own industry also deserves some blame. It is only very recently (in the last few years) that the security community has put the “secure by default” philosophy into practice. Previously, software offered confusing and insecure configurations which made HTTPS weak or easily broken. For years OpenSSL, the most widely-used SSL/TLS library, was shipped with default settings that were vulnerable, “rendering the middlebox[es] [which used OpenSSL] vulnerable.”
Hopefully this research has made it apparent that HTTPS interception is a major issue which must be addressed by the security community and the IETF, the organization which designs the TLS protocol.