Bot Tracker classifies every crawler that touches your site into three states — cite-worthy, neutral, or hostile — so you can welcome the ones that drive AI referrals and block the ones that just harvest your content.
The only tracker that links crawl activity to actual citation events and downstream referral traffic. Prove which bot visits produce business impact — and which are pure cost.
Powered by the open 3LA AI Bot Database of 200+ named crawlers, including GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider and CCBot.
Bot Tracker groups crawler activity so you can investigate useful discovery, routine collection and suspicious requests separately. Review the paths and request patterns behind a classification before blocking an entire crawler family. A policy change can affect legitimate access as well as unwanted scraping.
Keep the crawl-to-referral chain honest
A visit, a citation and a referral are different events. Relate them when supporting data exists, but do not treat every bot request as a generated answer or every citation as a click. Use this distinction to explain the difference between technical reach and measurable audience traffic.
Frequently Asked Questions
What is the three-state classification?
Bot Tracker classifies every crawler into cite-worthy (bots whose visits lead to AI citations and referral traffic), neutral (search crawlers and monitoring tools) or hostile (aggressive scrapers, content harvesters and bots violating robots.txt). The classification is based on PTR + IP-range verification and observed behaviour.
How does Bot Tracker link crawl to citation?
Bot Tracker connects three data points: which bot visited which page (crawl), whether the page was subsequently cited in an AI-generated answer (citation via Brand Analyzer) and whether users clicked through from that answer (referral via analytics). This bot-to-citation-to-referral chain proves which bot visits produce business impact.
What is the AI Bot Database?
The open 3LA AI Bot Database contains 200+ named crawlers including GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, Google-Extended, Bytespider, CCBot and many others. It is continuously updated as new AI crawlers emerge.
How does PTR + IP-range verification work?
For every hit, Bot Tracker performs a reverse DNS (PTR) lookup and checks the source IP against known ranges for each bot operator. This prevents spoofed user-agent strings from polluting your data — only verified crawlers are classified.
Can I generate a robots.txt from Bot Tracker?
Yes. One-click AI-aware robots.txt generator produces directives based on the observed crawlers and their classification. Allow cite-worthy bots, block hostile scrapers — with recommended entries for each.
What is the difference between Bot Tracker and Log Analyzer?
Bot Tracker classifies crawlers in near-real-time and links to citation events. Log Analyzer parses uploaded server access logs and produces a historical report with 90-day archive. Use Bot Tracker for live monitoring and Log Analyzer for historical analysis.
