The researchers uncover some interesting data about SHA-1 and passwords in general
CynoSure Prime, a group that classifies itself as a “password research collective” has reverse hashed 320-million passwords from Troy Hunt’s Have I Been Pwned database. The collective, working in conjunction with a pair or other researchers, came away with some interesting information while also commenting on the practicality of blacklisting such a lengthy list of leaked passwords.
Per CynoSure Prime’s post:
Blocking common passwords during account creation has positive effects on the overall password security of a website . While blacklisting 320m leaked passwords might sound like a good idea to further improve password security, it can have unforeseeable consequences on usability (i.e, the level of user frustration). Conventional blacklist approaches typically include the 10k most common passwords to limit online password guessing attack consequences. Until now, there has been no evidence to support which blacklist size provides an optimal balance.
If you’re not a regular part of the infosec community, the only reason you might know CynoSure Prime’s name is because of its works on the Ashley Madison breach a couple of years ago. Even then, CynoSure has only posted a handful of times since 2015. Regardless, the research is impressive. Here are some of the key takeaways.
Also, before we go any further, here’s a functional definition of hashing:
Hashing is the act of transforming a string of characters into a shorter, typically fixed-length value or key. It’s typically used to index and find items in a database, though it does also serve other functions as well.
More Dirt on SHA-1’s Grave
While as many as 15 different hashing algorithms appeared in Hunt’s database, the vast majority of the passwords were hashed with good old SHA-1. SHA-1 was officially outmoded a couple of years ago and then, just to remove any doubt, Google spent millions of dollars and conducted years of research that culminated in a SHA-1 collision last year—that really ended the discussion.
Which a quick aside, that’s terrifying, Google. The vast majority of us were willing to take the researchers at their word when they said SHA-1 was vulnerable. But you, you dedicated vast resources and brain power to create an outcome that was, to everyone else, just an abstraction at that point. For what? To prove a point? Terrifying.
Anyway, Google will be happy to know that SHA-1 is still very dead.
Out of the roughly 320 million hashes, we were able to recover all but 116 of the SHA-1 hashes, a roughly 99.9999% success rate.
CynoSure was also able to extrapolate further and resolve “nested” hashes, or hashes within hashes, to their plaintext forms. This table breaks down the layout of the different hashes found across Hunt’s data (images courtesy of CynoSure Prime):
Additional Information Contained in Hashes
One of the other issues that CynoSure found was that many of the groups that were originally storing this hashed data – Hunt’s data is compiled from high profile breaches – were also storing additional personal identifying information like email/password combos. Some even contained fragments of whole files.
For what it’s worth, Hunt never intended to include this information in his data release and has pledged to scrub it.
The fact that this information exists comes down to mistakes by the original owners with things like parsing.
Quick Hitter Facts from the Exercise
Here are some of the best quick hitter facts from CynoSure’s research:
- The longest password found was 400 characters
- The shortest password found was 3 characters
- 06& of all passwords were greater than 50 characters
- 67% of all passwords were less than 16 characters
This Was Not Challenging
Perhaps the most notable takeaway is just how powerful some of the tools on the market currently are. Using MDXfind and Hashcat on a quad-core Intel Core i7-6700K system, with four GeForce GTX 1080 GPUs and 64 gigs of memory, the researchers reverse hashed all but 116 of the hashes in under an hour.
What We Hashed Out (For the Skimmers)
Here are the highlights from today’s conversation:
- CynoSure Prime, a “password research collective” did a deep analysis of data made available by researcher Troy Hunt and reverse hashed nearly 320 passwords.
- SHA-1 did not fare well in this exercise, with researchers managing to reverse hash 99.99999% of the SHA-1 data.
- The average password is less than 16 characters.