(1 votes, average: 5.00 out of 5, rated)

Loading...

• Facebook
• Twitter
• LinkedIn
• Mail

June 14, 2024 Comments closed

Researchers Blocked 6 Trillion Requests From ‘Bad Bots.’ Here’s What They Found

in Beyond Hashed Out, Hashing Out Cyber Security, Monthly Digest

Nearly half of internet traffic identified by Imperva isn’t human; it’s bots, and most are up to no good. Here’s what they’re doing and how you can identify them and fight back…

Let’s start by stating that not all bots are bad. After all, Google’s search engine web crawlers make indexing your new web pages possible. But when “bad bots” engage in automated attacks against websites and/or APIs to steal, damage, and defraud businesses and consumers, then it becomes a huge problem that must be dealt with ASAP. 

Just how big of an issue are these bots? Imperva’s 2024 Bad Bot Report data, which focuses on OSI Model layer 7 (i.e., application layer) bad bot activity, shows that nearly half of the internet traffic observed via the company’s global network in 2023 stemmed from automated (i.e., non-human) sources. Bad bots represented nearly one-third of that traffic.

When we’re talking about nearly 6 trillion blocked bad bot requests in that period alone, it’s easy to see why they’re considered a significant threat. If each bad bot request counted as a separate second, those bad bot requests would add up to more than 189,000 years!

Knowing this, it’s time to explore the threats bad bots pose to organizations globally and what you can do to stop them.

Let’s hash it out.

A High-Level View of These Threats: What Imperva’s Data Tells Us

Bad bot activity is on the rise and doesn’t appear to be quitting anytime soon. Imperva’s data tells us that total bot activity peaked in 2014, when nearly 60% of all internet traffic was comprised of bots (36.3% good bots, 22.8% bad bots). Now, compare this with the internet activities of us lowly humans — human-based actions were at an all-time low, accounting for just 40.9% of all internet traffic that year.

Fast forward to 2019, when the combined good/bad bot activity dipped to just over 37%. But that activity has been growing steadily since then, with bad bots having the upper hand over good ones. In 2023, bad bots (32%) and good bots (17.6%) accounted for nearly half (49.6%) of all internet traffic!

But what are these bad bots capable of doing?

Primary Bad Bot Uses and Actions

In their 2024 Bad Bot Report, Imperva researchers identified bad bot-related activities as the following use cases (first three columns of the table below). We’ve broken these down these 10 use cases into four overarching categories (right column):

Bad Bot Problem	Definition	Business Impacts	Bad Bot Action Category
Price Scalping	Using bots to track and scrape product or service pricing information	Sales and revenue losses, decreased conversion rates, and brand reputational damages	Competitive Attacks
Content Scraping	Using bots to steal content and data from other sites	Loss of original content, SEO ranking drops, decreased site performance, and harm to brand reputation	Competitive Attacks
Account Takeover	Using bots to gain access to others’ accounts illegally	Increased support and fraud-related costs, negative publicity and reputational harm, and decreased performance	Account Security Compromises
Account Creation	Using bots to automate create fake accounts for misuse and fraudulent activities	Revenue losses, brand and reputational damages, and potentially bad decision-making due to poor data	Account Security Compromises
Credit Card Fraud	Using bots to brute force credit card info verification	Regulatory non-compliance, reputational harm, financial losses, increased support requirements, and loss of customer relationships	Financially Focused Attacks
Denial-of-Service	Using bots to overload legitimate sites, web apps, and services with traffic	Revenue losses, poor performance, customer relationship losses, and brand and reputational harm	Product or Service Availability Hijacking
Gift Card Balance Checking and Abuse	Using bots to get their hands on gift card balances	Fraud-related financial losses, increased support requirements, reputational harm, and potential customer relationship losses	Financially Focused Attacks
Denial of Inventory	Using bots to shopping cart “squat” to prevent legitimate purchases	Sales and revenue losses, increased shopping cart abandonment rates, decreased conversions, reputational harm, and potential loss of customer relationships	Product or Service Availability Hijacking
Scalping	Using bots to obtain services or goods to the detriment of legitimate customers	Poor user experience, downtime or performance issues, decreased average basket value (ABV), and reputational harm that could impact customer relationships	Product or Service Availability Hijacking
Seat Spinning	Using bots to “seat squat” airline tickets to keep legitimate users from buying them	Revenue losses and reputational damage that could impact customer relationships	Product or Service Availability Hijacking

Bad Bots + ATO = A Never-Ending Concern for Businesses

Account takeover (ATO) is one of the attack methods for targeting businesses favored by bad guys. This approach often involves attackers using bots to brute-force or credential-stuff their way into an account using lists of guessed or stolen passwords. Imperva’s data shows that over the past two years, ATO attacks increased 10% from 2022 to 2023.

As far as how often these attacks occur for businesses globally, the United States is leaps and bounds ahead of other countries. It ranks #1 as far as being targeted by the highest percentage of ATO attacks (40%). What can we say? While this rank isn’t something to be proud of, we’re not a country that likes to be outdone in virtually any category!

Are you wondering which industry experiences the most ATO attacks? Financial Services took the gold. But if you were to consider the industry that experienced the highest ATO malicious login ratio, then Business Services (38%) is the so-called “winner.”

Attackers <3 Your APIs… And This Can Impact Your Reputation & Bottom Line

Your authentication APIs are incredibly attractive targets for cybercriminals that increase your organization’s attack surface. More than two in five bot-based ATO attacks targeted application programming interfaces — likely because of the juicy data they provide access to. And a huge concern regarding API attacks relates to the use of automation.

Imperva’s data indicates that automation was used in 30% of the API attacks they analyzed in 2023. Automated attacks can be used for everything from brute force and credential stuffing attacks to DDoS attacks that overwhelm your API, disrupting services and impeding your legitimate customers.

DDoS Attacks Can Take a Hefty Chunk Out of Your Bottom Line

Data from Radware indicates that a distributed denial of service (DDoS) attack costs an average of $6,130 per minute as of August 2023. Now, consider that Zayo Group estimates the average DDoS attack lasted 68 minutes in 2023. This means that the average company forked over an average of $416,840 in DDoS-related costs that otherwise could have stayed in their bank accounts.

Now, imagine if a DDoS attack on your business lasted 24 hours. Based on that per-minute estimate, it would mean $8.8+ million in potential DDoS-related costs.

Consumers Increasingly Hold Businesses Accountable For Security Issues

According to Sift’s Q3 2023 Digital Trust & Safety Index data, 73% of consumers say that businesses are accountable for credential compromises stemming from ATO attacks. What makes matters worse for those organizations is that three in four of those customers indicate they’d never again do business with ATO-sullied brands.

So, knowing all of this bad bot data… what can you do to protect your business?

How to Battle Bad Bots: A 2-Pronged Approach

Bots are automated software programs that are designed to mimic human behaviors and actions. Unlike human hackers, bots are automated systems that virtually never stop. They don’t need to eat, they don’t need to sleep, and they never get bored.

However, there are things you can do to harden your organization and digital assets against automated attackers without impeding “good” bots. We’ll explore two primary approaches you can take to mitigate the risks of bad bots to your business.

1. Manage and Block Bad Bots From the Get-Go Using the Right Tools

Use Device, Network, and Website Monitoring & Analysis Tools

Robust firewalls can help you keep a close eye on your network and website traffic. It’s a good indicator that you have some potentially malicious bot-related activity if you notice any or all of the following:

• Significant jumps in traffic
• Increased bounce rates
• Specific IP addresses with record-high page view rates
• Increased traffic from IP addresses from outside your customers’ and other network users’ geographic regions
• Other related metrics

Web Application Firewalls (WAFs)

These tools are traditionally viewed as the go-to solutions for detecting and blocking bots. They’re useful for monitoring and detecting OSI Model layer 7 activity to identify and mitigate application layer attacks. However, relying on traditional WAFs may not be enough anymore due to the increasing complexity of these attacks.

Within the industry, we’ve been seeing movement toward web application and API protection (WAAP) solutions for bot mitigation, which expand upon traditional WAF capabilities to encompass greater protections.

Although Imperva’s research data focuses on application layer attacks, it’s important to note that bots can also be used to carry out network and transport layer attacks as well. Which brings us to the next firewall solution worth mentioning…

Next-Generation Firewalls (NGFWs)

Next-gen firewalls offer enhanced packet filtering and deep packet inspection (DPI). They’re great for monitoring and detecting threats on multiple OSI layers (3, 4, and 7). 

Using these firewalls, in addition to other third-party solutions and tactics, can give you a multi-layered approach that paints a clearer picture of what’s going on within your IT ecosystem.

• Security incident and event management (SIEM)
• Network traffic analysis (NTA)
• Network behavior analysis (NBA) or network behavior anomaly detection (DBAD)
• User and entity behavior analytics (UEBA)
• Encrypted traffic analysis / encrypted traffic intelligence (ETA/ETI)

Some of these tools use machine learning (ML) to enable you to analyze the traffic flowing between users’ clients and your server(s) without having to decrypt the sensitive data first. This enables you to keep an eye out for increased traffic stemming from outdated browser versions. (It’s a big red flag if you see a bunch of traffic coming from legacy browsers and devices.)

In particular, a combination of WAFS and NGFWs can help your business stay on top of ever-evolving threats at all three OSI model levels.

Implement Bad Bot Blocking and Mitigation Techniques

Creating a traditional server robots.txt file isn’t going to aid your fight against bad bots. However, it does help you limit good bots from crawling off-limits areas and sending too many requests, which can overwhelm your server if they get a little overzealous.

Here are a couple of techniques that can help mitigate bad bot traffic on your site and network:

• Move your login page to a non-traditional file path — Bad bots can’t hammer your login page if they can’t find it. Rather than using a traditional login page like /wp-login.php, rename the file path to something unique like /not-today-hackers.php or something else different that they aren’t likely to guess. By moving the end zone, they won’t be able to score a touchdown.
• Use honeypots to keep bad bots busy — Some researchers believe another way to mitigate DDoS attacks on your network’s legitimate systems involves implementing honeypots. By feeding them fake data, you keep the bots distracted and focused on a safe target. Additionally, you can use the data gleaned from these bot interactions to further harden your defenses against future attacks.
• Implement IP allowlisting and blocklisting — Traditional blacklists/blocklists are not as effective anymore. This is in part because evasive or increasingly sophisticated bad bots often use IP distribution techniques and you also risk blocking legitimate users whose IP addresses change regularly. However, allowlists/whitelists may still have anti-bot uses in some cases.    

Set API Request Rate Limits

One way to up your API security game is to restrict the number of requests that can be made to your network, website, or services. If any given IP address tries to make too many API call requests, then it’s going to either throttle its traffic or shut it down completely.

Furthermore, be sure to secure all of your organization’s APIs against common threats, including injection attacks. This statement applies to your website, apps, and online services. Check out OWASP’s Top 10 API Security Risks for more information on common API security risks and how to address them.

Set Login Failure Rate Limits

High rates of login failures are a big red flag for bot-related activities. To help prevent brute force and credential stuffing attacks, limit the number of times a user, device, or IP address can attempt to log in.

Use a Content Delivery Network (CDN) and DNS Failover for Traffic Distribution

Of course, your business needs to be prepared to handle the traffic surges that are common with DDoS attacks that can cause failing servers and downtime. Using a secure content delivery network (CDN) is a great way to combat these surges.

A CDN takes all of the inbound traffic your site, app, or service receives and distributes it across many servers around the world. Likewise, a DNS failover solution updates your DNS records to direct traffic to the fastest available instance.

This way, the responsibility doesn’t fall on only a handful of devices that attackers can overwhelm and result in downtime.

2. Eliminate Vulnerabilities That Bots Like to Exploit

Now, let’s explore what you can do to harden your security against common exploits that bad bots like to target and use to their advantage.

Regularly Scan for Vulnerabilities

The first task on this list is scanning for vulnerabilities on your network, endpoints and website. These are weaknesses in your organization’s defenses that bad guys and their drones can focus on. Think of it like having holes in your castle wall that they can slip through.

Cybercriminals love to go after the lowest-hanging fruit (i.e., insecure businesses that are the easiest to attack using the least amount of effort). Examples of the insecurities they love to target include legacy apps and firmware, outdated themes, security misconfigurations, and coding-related exploits. 

Lock Down Authentication

Like onions and chocolate cake, account security is all about the layers. Having robust authentication mechanisms on your side is one of your biggest defenses when it comes to controlling access to your site, web apps, and services. Some examples of these approaches include:

• Adopting a zero-trust architecture when applicable — This self-explanatory approach is all about requiring continuous authentication. Rather than a user signing in once and having free reign, you’re requiring the user to authenticate throughout their session.
• Use authentication challenges to control access — CAPTCHAs are useful (albeit annoying) security measures to help mitigate bots. However, another approach you can take is a simple UX trick: stack two elements on a page so that human users click on the upper element, whereas bots wind up “clicking” both layers simultaneously because they can’t differentiate the layers.
• Rolling out multi-factor authentication — Require users to provide multiple authentication factors in order to gain access to access your systems. When MFA is enabled, a user must provide two or more of the following authentication factors:
  • Something you know (such as a password or PIN)
  • Something you have (e.g., a push notification on your phone or a USB token)
  • Something you are (a biometric such as a fingerprint or face ID scan)

Set Stringent Password Requirements

Educate and encourage employees to use secure passphrases in lieu of traditional passwords. A passphrase like WordleGam3r$Un1te! is easy for you to remember but difficult for a bot to guess.

According to Hive’s 2024 Password Table (shown below), this 18-character passphrase containing upper and lowercase letters, numbers, and symbols would take 19 quintillion (qn) years (19,000,000,000,000,000,000 years) to brute force using the brcrypt hashing algorithm (and the specified hardware noted in the table below).

Image source: Hive Systems.

Implement Robust Password Policies and Rules

There is more you can do for password security than setting strict password requirements. You also can implement account password reset limits. This way, if one or more bots repeatedly try to reset a password for a specific account, it’ll trigger an account lock that shuts down the attempts.

Use Cryptographic Password Storage Protections

Storing your network users’ and customers’ passwords in your database in any form — plaintext or encrypted — is a big no-no. (Plaintext passwords can be stolen via eavesdropping and encrypted ones can be exposed if the decryption key is compromised.) Instead, store salted and peppered password hashes.

Applying a cryptographic hash function to your passwords gives you a way to allow users to authenticate without their plaintext passwords needing to be stored (and potentially exposed) in your database. A one-way hashing algorithm (such as bcrypt) generates a fixed-length hash value. It’s this hash value (i.e., hash digest) that you should store in your database instead in place of plaintext or encrypted passwords.  

Looking for More Info? Check Out These Industry Resources

• Open Web Application Security Project (OWASP) — This OWASP Foundation offers many great resources through its OWASP Automated Threats to Web Applications Project to help web app owners fight back against automated attackers.  
• CISA, FBI and Multi-State Information Sharing & Analysis Center (MS-ISAC) — The joint guide “Understanding and Responding to Distributed Denial of Service Attacks” addresses many of the needs and concerns surrounding DDoS attacks. It breaks down three common types of DDoS and DoS attacks and outlines best practices you can implement.
• Imperva — This security company has some great information on basic and advanced bot mitigation techniques.

• #Bad Bots