What Is a Bot? Good Bots, Bad Bots and Bot Management
A bot is software that runs automated, repetitive tasks online far faster than a person. Some bots index your site for search. Others test stolen passwords, scrape prices or fake those sign-ups.
The Problem: Half of Your Traffic May Not Be Human
Most owners assume the visitors in their analytics are people. Imperva’s 2025 Bad Bot Report, based on 2024 data from its global network, found automated traffic made up 51% of all web traffic. Bad bots alone were 37% of internet traffic, up from 32% in 2023. Good bots were 14%.
That changes how you read every business metric. Conversion rates drop because the denominator is inflated. Ad budgets pay for clicks no human made. Login pages take thousands of guesses a day. None of it looks like a “hack,” so it often goes unnoticed until a customer complains that their account was emptied.
What Is a Bot?
A bot (short for robot, also called an internet bot or software robot) is a program that performs automated tasks over a network without a person driving each step. It follows rules or a script, repeats them at machine speed, and can run on one server or thousands of devices at once. Bots are neutral tools. Intent decides whether one is good or bad.
What Is a Bot in Cyber Security?
In security, “bot” usually means automated traffic that hits your websites, apps and APIs. The question is never “is this a bot?” alone. It is “is this bot allowed, and is it doing what it claims?” A scraper pretending to be Googlebot is the classic trick.
How Do Bots Work?
A bot sends requests to a site or application just as a browser does, but under program control. Simple bots use scripts that fetch pages or submit forms. Advanced bots run real browser engines, rotate through residential IP addresses, solve or outsource CAPTCHA challenges and mimic mouse movement. Imperva’s 2025 report found API-directed attacks made up 44% of advanced bot traffic, because APIs return clean data with no page to render.
Good Bots vs Bad Bots: What Is the Difference?
The difference is permission and purpose. Good bots identify themselves honestly, follow your rules and do something you benefit from. Bad bots hide or fake their identity and take value from you.
| Good bots | Bad bots | |
|---|---|---|
| Examples | Search engine crawlers, uptime monitors, feed readers, your own chatbot | Credential stuffing, scraping, scalping, spam, click fraud |
| Identity | Declared and verifiable | Hidden or spoofed |
| Respects robots.txt | Usually | No |
| Effect on business | Visibility, monitoring, service | Fraud, cost, downtime, data loss |
Common Types of Good Bots
Search engine crawlers such as Googlebot and Bingbot index pages so customers can find you. Monitoring bots check uptime and performance. Feed bots pull your blog or product updates into other services, and chatbots answer customer questions on your behalf.
Gray Area: AI Crawlers and Aggressive Scrapers
AI crawlers collect content to train or feed AI tools. They may be legitimate companies, yet you may not want their load or their use of your content. The robots.txt standard (IETF RFC 9309, 2022) lets you state crawl rules, but compliance is voluntary. Decide per crawler: allow, limit or block.
Types of Malicious Bots
The OWASP Automated Threats to Web Applications project catalogs 21 named automated threats. These are the ones businesses meet most.
Credential Stuffing and Account Takeover Bots
These bots replay leaked username and password pairs across your login page. Any customer who reused a password can lose their account, which leads to account takeover (ATO), fraud and support costs.
Web Scraping and Price Scraping Bots
Scrapers copy content, product data and prices. A competitor or reseller can undercut you within minutes of a price change.
Scalper, Inventory and Carding Bots
Scalpers buy limited stock to resell. Inventory-hoarding bots hold items in carts so real buyers see “sold out.” Carding bots test stolen card numbers with small purchases, leaving you with chargebacks.
Spam, Click Fraud and Fake Account Bots
These bots post spam, fill forms, create fake accounts to abuse promotions and click your paid ads to drain budget.
DDoS Bots
DDoS (distributed denial-of-service) bots flood a site or API with traffic until legitimate users cannot get through.
Bot vs Botnet
A bot is a single automated program. A botnet is a network of many devices infected with malware and controlled remotely by an attacker, usually without their owners knowing. Botnets supply the scale and the spread of IP addresses that DDoS and credential stuffing campaigns need. Every botnet is made of bots, but most bots are not part of a botnet.
How to Tell If Website Traffic Is a Bot
Look for patterns a human would not produce:
- Very high request rates from one IP address or a narrow range
- Traffic at odd hours with near-zero time on page
- Many failed logins across different usernames
- Sudden spikes in account creation or cart activity with no matching sales
- User agents claiming to be a search engine that fail verification
Google documents how to verify Googlebot with a reverse DNS lookup followed by a forward lookup. If a “Googlebot” fails that check, treat it as an impostor.
What Is Bot Management?
Bot management is the practice of identifying automated traffic, separating good bots from bad ones and applying the right response to each. Unlike a simple blocklist, it allows helpful crawlers, slows or challenges suspicious visitors and blocks known abuse, so customers and search engines keep working while attackers lose their advantage.
How Does Bot Management Work?
Traffic is scored as it arrives. Each request gets a likelihood of being human, good bot or bad bot. Policies then decide what happens, and the system keeps learning from outcomes.
Bot Detection Techniques
- Device and browser fingerprinting: checks whether the browser is what it claims to be.
- Behavioral analysis: looks at mouse movement, typing rhythm and navigation flow.
- Reputation and verification: uses IP reputation and verified good-bot lists.
- Machine learning: spots new patterns across large traffic volumes.
Bot Mitigation: Block, Challenge, Rate Limit or Allow
Blocking suits known bad actors. Challenges, such as a CAPTCHA, suit uncertain traffic. Rate limiting caps how fast anyone can hit a login or search endpoint. Allowing applies to verified good bots. The best results come from matching the response to the risk of the page.
Basic Steps to Stop Bad Bots for Small and Midsize Businesses
You do not need an enterprise platform on day one. These steps cut the worst abuse quickly:
- Protect the login. Turn on MFA for customer and staff accounts and rate-limit failed attempts per account and per IP.
- Rate-limit expensive endpoints. Search, checkout, password reset and APIs should all have caps.
- Turn on your existing tools. Most content delivery networks and web application firewalls (WAFs) include basic bot rules that are often switched off.
- Watch three numbers weekly. Failed logins, new accounts per day and checkout attempts without payment.
- Write your robots.txt rules deliberately. Decide which crawlers, including AI crawlers, you allow.
A useful threshold: if failed logins exceed successful ones on any day, assume credential stuffing and act that day.
When to Choose a Bot Management Solution
Move to a dedicated solution when bots affect revenue, not just logs. Ask vendors:
- How do you detect bots that run real browsers and residential IPs?
- Do you protect APIs and mobile apps, not just web pages?
- How do you verify good bots so search ranking is not harmed?
- What is your false positive rate, and how do customers appeal a block?
- What does reporting show my team each week?
Is bot management the same as a WAF? No. A WAF inspects requests for attack payloads such as SQL injection. Bot management judges who or what is sending requests, even when each request looks valid. Many products bundle both.
Bots and the Law in the US and UK
In the US, the Better Online Ticket Sales (BOTS) Act of 2016 bans using bots to get around ticket purchase limits, and the Federal Trade Commission (FTC) enforces it. The Computer Fraud and Abuse Act can apply when bots access systems without authorization. In the UK, the Computer Misuse Act 1990 covers unauthorized access, and the UK GDPR applies when bots harvest personal data. Scraping law depends on facts and terms of service, so confirm specifics with counsel.
What Is a Bot Costing You? Measure Before You Block
So what is a bot to your business? Bots are not one problem. They are good helpers, gray-area crawlers and outright fraud tools sharing the same front door. This week, pull your failed-login and new-account numbers for the last 30 days and compare them with real sales. That one chart tells you whether you need rate limits or a full bot management program. If you want a second opinion on the numbers, Cyber Security Solutions Ltd offers a free cyber security assessment through our contact page.
FAQs
No. Many bots are essential. Search engine crawlers make your site findable, monitoring bots alert you to outages, and chatbots handle customer questions. Imperva's 2025 Bad Bot Report found good bots made up 14% of internet traffic. The goal of bot management is to keep these helpful bots working while stopping malicious ones.
Imperva's 2025 Bad Bot Report, based on 2024 data, found automated traffic made up 51% of all web traffic, the first time in a decade it exceeded human traffic. Bad bots accounted for 37% and good bots for 14%. Your own site's mix will differ by sector, so measure your own traffic.
No. A CAPTCHA stops simple scripts, but advanced bots use solving services, AI models or low-paid human workers to get past challenges. CAPTCHAs also frustrate real customers. Use them as one layer, triggered only for suspicious traffic, alongside rate limiting, MFA and behavioral detection rather than as the only defense.
Not quite. A web application firewall inspects requests for malicious payloads like SQL injection or cross-site scripting. Bot management decides whether the sender is a human, a good bot or a bad bot, even when every request looks valid. Credential stuffing passes most WAF rules, which is why the two work best together.
Warning signs include a slow device, high network activity when idle, unknown processes, a fan running constantly, and contacts receiving spam you did not send. Run a full scan with up-to-date endpoint protection, update the operating system and change passwords from a clean device. On business networks, ask IT to check firewall logs.
Using a bot is legal in most cases. Search engines, monitors and chatbots are lawful. A bot becomes illegal when it is used for unauthorized access, fraud, or breaking specific rules such as the US BOTS Act on ticket purchases. In the UK, the Computer Misuse Act 1990 covers unauthorized access by any means.
