(No Ratings Yet)

Loading...

• Facebook
• Twitter
• LinkedIn
• Mail

July 31, 2017 1

Why Chrome Tells You That HTTPS Sites are “Secure”

in Everything Encryption

Seven Options Were Tested To Accompany the Padlock Icon, Including “Private” and “Encrypted”

Last year in December, Chrome 55 introduced the “Secure” label to indicate a valid HTTPS connection, joining the padlock icon which has been used by browsers for decades.

Google Chrome is undeniably the innovator and leader in browser security UI. Their team has been making a multifaceted effort to improve HTTPS by simplifying icons (remember that yellow “warning” triangle that used to be in Chrome?), rewriting browser errors in plain English, and warning users when they are not secure.

The new “Secure” text was part of a revamp of Chrome’s security indicators based on research conducted by their engineering team. The goal was to make their indicators easier to understand for end-users.

It was partly motivated by a family member of Chrome’s security engineering team, who couldn’t understand why her browser displayed a purse icon in the address bar.

When you compare them side by side, the padlock does bear strong similarities to ‘shopping bag’ icons used on many e-commerce sites:

 

“You’re shopping at Apple Inc.”#https pic.twitter.com/RAZhOfcpFm

— Michal Špaček (@spazef0rze) September 27, 2016

//platform.twitter.com/widgets.js

 

But for some, the “Secure” label has been controversial.

As a result of a number of factors – including the browser-led push for HTTPS and the rising popularity of automated and free certificates – there are more malicious sites than ever using SSL. The fear is that users will fail to understand the nuances of what their browser is telling them – that their connection is “Secure”- and conflate that with the site being safe overall (free from other threats, such as malware) and/or legitimate (e.g. a “real” site instead of a phishing site).

When Chrome’s team set off to see how it could make their icons look less like a double-handle tote, it tested text labels alongside the icons to “aid user comprehension of security indicators, particularly for new Internet users who do not have preexisting expectations of icons.”

Given that the “Secure” label has been in Chrome for nearly 9 months now and is still drawing criticism, I wanted to take a deep-dive into the research and testing behind this change, and the alternative options Chrome tested which failed to make the cut.

Collecting Data

Chrome’s team presented its work last year at the Symposium on Usable Privacy and Security (SOUPS).

The paper covers how Google overhauled all of its indicators, but we want to focus on just one part: The “Secure” label that now accompanies every valid HTTPS connection in Chrome.

Seven different text labels were tested in combination with the padlock icon. Those labels were: “https,” “Private,” “Secure,” “Safe,” “Encrypted,” “Secure and Private,” and “Secure site.”

Through an online survey, Chrome’s team collected responses from 6,462 users located in the United States.

The survey showed the users “a mock browser screenshot that included an icon, string, and blurred URL,” and asked three questions, designed to cover three different categories: perception of safety, the identified threat, and the user’s resulting action.

Here is our own mock-up example, similar to what users were shown:

The survey questions were:

1. If you saw this browser page, how safe would you feel about the current website?

• Not at all safe
• A little safe
• Somewhat safe
• Very safe
• Extremely safe

2. If you saw the below icon and message in the browser’s address bar, that might mean that someone might [Could select multiple options or “None of the above”]…

• Try to put a virus or malware on your PC
• Modify the content of the page
• Have created a technical bug on the site
• Steal the things you read and type
• None of the above

3. If you came across a site in your browser and saw this in the address bar, how would you most likely proceed?

• I’d browse normally
• I’d leave the site
• I wouldn’t enter any credit card details
• I’d look for more information about the site
• I’d browse quickly, then leave

Users were shown one of the seven text candidates and answered all three questions about the same design.

The Results

Chrome’s team had already decided on keeping the padlock because of its familiarity. The goal now was to decide which of the seven candidates would be the best accompanying text. Users were asked to assess their overall reaction of the above mock-up – so they were judging the combination of both the icon and text.

Here are the results from the first question: “If you saw this browser page, how safe would you feel about the current website?”

When we look at these numbers it is fairly hard to come to a conclusion. A number of the candidates appear to have performed similarly.

To make it easier to understand their performance, I assigned numerical value to each response, from 1-5, and then average them. For example, each “Not at All” response would receive 1 point, and each “Extremely” response would receive 5 points. Each candidate could then be compared by their overall score.

This is similar to a Likert Scale. Statisticians are screaming at this moment because I am about to commit two cardinal sins. For one, the responses chosen by Google do not match a Likert Scale exactly. Additionally, a Likert Scale is an ordinal scale, not an interval scale, which means a mathematical average is not very appropriate.

However, this is not an article for an academic journal, and interpreting data from a Likert scale as interval data does have some precedent for a more general audience.

That aside, when we score the 7 candidates we can get a better idea of how each option performed overall:

“HTTPS” and “Secure” tie for first, followed by “Safe” and “Secure Site,” which tied for second. “Private” comes in last, and “Encrypted” only beats it by a fraction of a point.

Chrome’s research team has its own methodology for picking a winner here: “‘Secure’ yielded the highest number of respondents who felt that the website was at least somewhat safe [40%] and the lowest number of participants who felt not safe at all [12%].”

If the goal is to minimize the number of users that felt “Not At All” safe and maximize the number of users who feel “Somewhat” safe, then “Secure” is the clear winner. No other candidate performs well in both scenarios.

Earlier on I said each survey question was intended to assess a different aspect of effectiveness. This first question was assessing the perceived security of the page.

We already have two early front-runners: “HTTPS” and “Secure.”

On to the second question: “If you saw the below icon and message in the browser’s address bar, that would be that someone might.”

With this question users could either select multiple options or “none of the above”. This question was designed to test what types of threats users thought the icon was designed to indicate. For the HTTPS icon, users should be able to differentiate between threats and identify the benefits provided by a secure HTTPS connection.

We would want to see users indicate the site could have a virus/malware (“Malware”), and that the site could have a technical bug (“Bug”), since HTTPS addresses totally different threats. They should not have indicated that their information was at risk (“Steal”) or that the page could be modified (“Modify”) since these are among the primary benefits of HTTPS, and would indicate they are getting the opposite of the intended meaning.

“None” could indicate poor comprehension of the icons or a mistaken belief that the indicator means a site is overall “safe” to use with no threats of any kind. A high percentage of respondents choosing “None” is bad.

Here were the results:

All seven candidates had a high percentage of users selecting “None.” Chrome’s team wrote that “this suggests that our indicators are broadly perceived as [overall] security indicators, not specifically as connection security indicators.”

This is significant. It means all the text candidates they chose (or the padlock icon on its own) is strongly associated with a “safe” site. This is a common misunderstanding browsers want to avoid, because from a technical standpoint, HTTPS makes very specific and narrow security assurances and is not a ‘cure-all’. For example, an HTTPS website can serve malware as easily as an HTTP website.

It is hard to evaluate the results of this question because there is no way to create a scale. There are (at least) two different ways to judge the best performance here:

1. Which options had the lowest combined percentages of “Steal” and “Modify,” as those are the two threats a user should not be concerned about on an HTTPS page
   .
2. Which options had the lowest percentage of “none of the above” responses?

The problem here is that no indicator performed well under both of those conditions. In fact, the candidates which performed well by our first metric performed the worse for the second, and vice versa.

By the first metric, “HTTPS,” “Secure,” and “Secure Site” performed the best, in that order. There is a noticeable gap between those three candidates and the rest. The next-best performing option trails by 5%. “Private” performed the worst.

However, by the second metric, “Private” performed the best with 51% of users selecting “none,” and “HTTPS” and “Secure” performed the worst with nearly 2/3rds of users thinking none of the threats applied.

“Private” also appears to be the least misleading. More users were able to identify that a site labeled “Private” could still have malware or a technical bug, two threats which HTTPS cannot protect against. This suggests that “Private” was the least likely to over-promise the security benefits it was providing.

Chrome’s team noted that “respondents were most likely to trust a page with the ‘https’ and ‘secure’ strings.” Those two strings did have the best association with the security benefits of HTTPS.

We could see all the options as failing given that more than half of respondents believed these indicated there were no threats.

Finally, question 3: “If you came across a site in your browser and saw this in the address bar, how would you most likely proceed?”

This question was designed to see what a user would do in response to seeing the secure connection indicator. The authors did not identify an ideal action here, but browsing normally (“Normally”) appears to be it.

We would not want users to immediately leave the website (“Leave site”) or browse quickly and then leave (“Quickly”). We can imagine the type of impact this could have on e-commerce and bounce rates if a secure connection was misleading users into navigating away from a page.

Once again, “Private” comes in last here. It had the highest percentage of users claiming they would leave the site (28% tied with “Encrypted”). Only 25% said they would browse normally if they say that combination of padlock and text – an absolutely abysmal percentage.

“Secure” had the lowest percentage of users saying they would leave the site, followed by HTTPS. Those two candidates also had the highest percentage of users claiming they would browser normally, though HTTPS wins that measure by a much larger margin.

Unexpectedly, “HTTPS” had the smallest percentage of users claiming they would not enter a credit card. Less technical candidates “Safe” and “Secure site” performed almost as well. “Secure” and “Private” performed poorly here.

Finding a Winner

We have looked at the data – now it’s time to find the overall winner. Of the seven candidates, there were some clear patterns throughout the survey.

Seeing as “Secure” now lives in the address bar of millions of Chrome users, we obviously know what Chrome’s team thought. In its paper, the team wrote: “we find ‘secure’ and ‘https’ are the most promising companions to a green lock icon.”

After looking at the data, they are right.

“Private” didn’t fare well – performing the worst nearly across the board. I do have to give “Private” credit for not over-promising protection – users had better accuracy identifying that a site labeled “Private” could have malware or bugs. But that alone cannot redeem its poor performance in most of the tested metrics.

One candidate which we have not talked about much is “Encrypted.” From a technical standpoint, this is one of the more accurate options. In the survey results it was fairly middle of the road – generally performing worse than “HTTPS,” especially in key areas of user action (question 3). It blurs the line for general comprehension. While it is not technical jargon like “HTTPS,” it’s unlikely that most users have a good understanding of how encryption works.

Interestingly, 31% of responses incorrectly thought the page could be modified or their information stolen when labeled “Encrypted,” compared to only 24% for HTTPS. This challenges the assumption that most internet users don’t understand what “HTTPS” is.

“Safe” would have been a poor choice because of its overly-broad meaning. The entire issue here is that with the general way we use “secure” and “safe,” they are synonymous, but on the internet they are entirely unrelated concepts.

Browser security and UI teams have been intentionally trying to separate the idea that a secure connection means a site is overall safe to browse. Ironically, “Safe” performed well for threat identification (question 2).

“Secure and Private” and “Secure Site” would both pose issues due to their size. Browsers already have a hard with proper URL display, and long text strings accompanying every secure page would not help. That alone would probably not preclude them, but they also failed to show any outstanding results in the survey.

That brings us back to the two candidates that Chrome’s team identified: “HTTPS” and “Secure.”

Before this redesign, Chrome already displayed “https://” in the address bar when a connection is secure. This is an indicator that has existed for years and may already be understood by users. Some of the survey results for “HTTPS” were promising – especially compared to less technical candidates – but given how long it has been around its recognition is lower than it should be.

Chrome’s team “preferred ‘secure’ because it is less technical.”

The incarnation we see today in Chrome is actually slightly different than what was tested. We have both – the “Secure” label and “https://” in the URL. The data shows that “Secure” and “HTTPS” were both strong performers, so having both may be covering more ground and helping user comprehension and perception. It also connects the two concepts which may help build understanding of the terms.

In the future we may see the “https://” scheme removed as Chrome tries to simplify its UI.

Helpful or Harmful?

At the top of this post I said that the “Secure” label was controversial. Even after looking at the data, the question remains:

Is it possible that the “Secure” label is unintentionally harming users?

Chrome’s study did not attempt to answer that question. While we can draw some general conclusions about user perception from this study, it would be misleading to make such a specific claim from such broad evidence.

It is undeniably true that some portion of users interpret the “Secure” label as meaning a site is totally safe and legitimate, and that they are making unsafe choices as a result. But we simply don’t know how big that proportion of users are.

It is also undeniably true that the “Secure” indicator – and it’s polar-opposite cousin, “Not Secure” – are encouraging more websites to adopt HTTPS. And that these text indicators are helping and educating some portion of users who aren’t well served by the padlock icon. We also don’t know how big that proportion is.

The reality is that one can’t conclusively make such a claim because it has not been studied yet.

But the goal of this article was not to convince you if the “Secure” label is good or bad, only to illuminate the reasoning behind its use and how it was selected. As the data shows us, commonly suggested alternatives such as “Encrypted” and “Private” were tested and had some major drawbacks, which, for Chrome’s team, negated their strengths.

Ultimately, Chrome’s goal is to treat HTTPS as the default state of the web, and the “Secure” label (and the padlock) will be removed in favor of stronger warnings for HTTP sites. So even if you believe the “Secure” indicator is the worst thing to happen to user security, you can relish in its inevitable death.

• #Google
• #Google Chrome
• #HTTPS
• #Security Indicators
• #SSL/TLS