Not every visit to your website comes from a person. Search engine crawlers, uptime monitors, link previews, scrapers and spam tools all load web pages, and together they make up a large share of internet traffic. Some of this bot traffic is useful, some is harmless and some distorts your statistics or wastes server resources. This guide explains the main kinds of bots, how counters separate them from human visitors and how to recognise suspicious traffic in your reports.
What counts as a bot?
A bot is software that requests web pages automatically. The term covers very different things:
- Search engine crawlers such as Googlebot and Bingbot read your pages so they can appear in search results. You want these.
- Advertising and verification crawlers check pages that show ads, for example to decide which ads are relevant.
- Link preview bots from messaging apps and social networks fetch a page to build a preview card when someone shares your link.
- Monitoring tools check every few minutes that your site is online.
- SEO and research tools crawl the web to build link databases and rankings.
- Scrapers copy content or prices, sometimes aggressively.
- Spam and attack tools probe for vulnerable login pages, submit forms or try to plant fake referrers in your statistics.
How bots affect visitor statistics
Many simple bots never run JavaScript, so they do not appear in JavaScript-based counters at all. They are visible in server logs, which is one reason hosting statistics often show far more visits than analytics tools. Bots that use a real browser engine do run scripts, and those can appear as page views unless the counter recognises them.
Well-behaved bots announce themselves through their user agent, the identification string every browser sends. GNRCounter compares the user agent of each request with a list of known crawlers, including the major search engines, and records matching requests separately as bot activity rather than as visitors. That keeps your visitor and page-view figures focused on people.
Bots that disguise themselves as ordinary browsers are harder to detect for every tool. They are a minority on most small sites, but they can produce strange patterns, described below.
Signs of suspicious traffic in your reports
- A sudden spike with no obvious cause: you did not publish anything, no newsletter went out and no site linked to you, yet visitors doubled for a few hours.
- Many visits to a single page at regular intervals: people do not arrive every 60 seconds exactly; scripts do.
- An unusual country: a burst of visits from a country where you have no audience and no content in its language.
- An odd browser or operating system mix: a large share of very old browser versions, or unusual screen sizes, appearing suddenly.
- Unfamiliar referrers: domains you have never heard of, often with names that invite you to click, appear as traffic sources. This is called referrer spam; the goal is to make site owners visit the spammer’s website.
- Pages that do not exist: requests for admin or login paths you never created show up in server logs.
What to do about referrer spam
The most important rule is simple: do not visit unknown referrer domains from your statistics. Many exist only to attract clicks, and some host unwanted software. Treat them as noise in your reports. Look at your genuine sources, such as search engines, social networks and sites you recognise, and judge your traffic by those. Our traffic sources guide explains how to read the referrer list.
Protecting your site from harmful bots
A counter measures traffic; it does not block it. If bots cause real problems, such as slowing your site or flooding forms, the solutions sit at other layers:
- Keep software updated. Most automated attacks target known vulnerabilities in old versions of content management systems and plugins.
- Protect forms. Use honeypot fields, rate limits or a privacy-respecting challenge on contact and registration forms.
- Use a web application firewall or CDN bot protection. Many hosting providers and CDNs offer bot management that blocks known bad traffic before it reaches your server.
- Use robots.txt for well-behaved crawlers. It tells legitimate bots what not to crawl. Malicious bots ignore it, so it is not a security tool.
- Do not block search engines by accident. Blocking Googlebot or Bingbot removes your pages from search results.
Never buy or generate fake traffic
Services that sell “website traffic” usually deliver bots or low-quality automated visits. They inflate counters but bring no readers, no customers and no useful data. Worse, if your site shows advertising, artificial impressions and clicks violate ad network policies and can lead to your account being closed. The same applies to refreshing your own pages or using auto-refresh tools to raise a visible total. A smaller, honest number is far more valuable than a large meaningless one.
Keep your own visits out too
Your own visits are not bots, but they distort small sites in a similar way. Exclude your browser in GNRCounter by opening a tracked page once with ?gnrc-hideme=1 at the end of the address. To undo it, use ?gnrc-hideme=0. Repeat this on each device and browser you use.
Frequently asked questions
Does Googlebot appear in my visitor count?
Known search engine crawlers identify themselves and are recorded separately from visitors, so they should not inflate your visitor figures.
Why does my hosting panel show so many more visits?
Server logs include every request from every bot, including those that never run JavaScript. They are useful for capacity planning, not for counting readers.
Can bot traffic hurt my search rankings?
Normal crawler activity is part of how search works. Problems arise only if aggressive bots slow your site down so much that real visitors and search crawlers have trouble loading it.
Should I block all bots?
No. Search engines, link previews and monitoring tools are useful. Focus on bots that cause measurable harm, and block them at the server or CDN level.
Next steps
Review your referrers and countries for the last month and note any unusual spikes. If the patterns look normal, you can trust your trends. To understand why tools report different totals, read why analytics numbers differ; to understand the remaining figures, see sessions, visits and users explained. Browse more tutorials under all guides.