Bot Detection

AttractOS detects and tracks 40+ AI bots from OpenAI, Anthropic, Google, Meta, and other major AI companies. See which crawlers are indexing your content, how often they visit, and which pages they access most. Bot detection works automatically via both client-side script and server-side Cloudflare integration.

Tracked Bots

AttractOS recognizes the following AI bots and crawlers. The list is continuously updated as new bots emerge.

OpenAI

GPTBot Training & browsing
ChatGPT-User User-initiated browsing
OAI-SearchBot Search feature

Anthropic

ClaudeBot Training
Claude-Web Web access

Google

Google-Extended AI training
Googlebot Search indexing

Microsoft

Bingbot Search indexing
Copilot AI assistant

Perplexity

PerplexityBot Answer engine

Meta

FacebookBot Content preview
Meta AI AI training
Meta Fetcher Content fetching

Apple

Applebot Siri/Search
Applebot-Extended AI training
SiriBot Siri suggestions

Amazon

Amazonbot Alexa/Search

xAI

GrokBot Grok training
Grok Grok training

ByteDance

Bytespider AI training

Cohere

Cohere AI training

Mistral AI

MistralBot AI training

Stability AI

StabilityBot AI training

Hugging Face

HuggingFaceBot Dataset

AI21 Labs

AI2Bot AI training

You.com

YouBot Search/AI

Brave

BraveBot Search

DuckDuckGo

DuckDuckBot Search

Common Crawl

CCBot Open dataset

Diffbot

Diffbot Knowledge graph

Semrush

SemrushBot SEO analysis

Ahrefs

AhrefsBot SEO analysis

Moz

Rogerbot SEO analysis

Majestic

MJ12bot SEO analysis

Screaming Frog

Screaming Frog SEO audit

Yandex

YandexBot Search

Baidu

Baiduspider Search

Sogou

Sogou Search

Qihoo 360

360Spider Search

Neeva

NeevaBot Search

Detection Methods

Client-Side (JavaScript)

The tracking script can detect bots that execute JavaScript. Limited effectiveness since most bots don't run JS.

Partial coverage

Server-Side (Cloudflare)

Cloudflare Logpush sends every request's User-Agent to AttractOS, catching all bots regardless of JS support.

Recommended

For complete bot visibility, we recommend setting up Cloudflare Logpush. This captures bots that block JavaScript execution.

Understanding Bot Data

For each bot visit, AttractOS records:

  • Bot name — The identified crawler (e.g., GPTBot, ClaudeBot)
  • Company — The organization operating the bot
  • Page path — Which page was crawled
  • Timestamp — When the crawl occurred
  • Country — Geographic origin of the request (via Cloudflare)

Dashboard Metrics

Your dashboard shows:

  • Total bot visits — All AI crawler hits in the selected period
  • Bot breakdown — Which bots visit most frequently
  • Top crawled pages — Your most visited content by AI
  • Trends — How bot traffic changes over time

API Access

Query bot data programmatically via the API:

# Get bot breakdown for a site
curl -X GET "https://attractos.com/api/v1/sites/{siteId}/stats?start=2024-01-01&end=2024-01-31" \
  -H "Authorization: Bearer YOUR_API_KEY"

# Response includes:
{
  "totals": {
    "bot_visits": 1847,
    ...
  },
  "bot_breakdown": [
    { "bot_name": "GPTBot", "bot_company": "OpenAI", "count": 523 },
    { "bot_name": "ClaudeBot", "bot_company": "Anthropic", "count": 312 },
    { "bot_name": "PerplexityBot", "bot_company": "Perplexity", "count": 289 },
    ...
  ]
}

Frequently Asked Questions

How does AttractOS detect bots?
AttractOS identifies bots by their User-Agent string. When a request comes in, we match the User-Agent against our database of known AI bot patterns. This works both client-side (via our script) and server-side (via Cloudflare Logpush).
What if a bot isn't on this list?
We continuously update our bot database as new AI crawlers emerge. If you notice a bot we're missing, email us at [email protected] and we'll add it within 48 hours.
Can I block specific bots while still tracking them?
Yes. AttractOS only tracks bots—it doesn't block them. Use your robots.txt file or server-side rules to block bots. AttractOS will still show you which blocked bots attempted to access your site (via Cloudflare Logpush).
Why do some bots have multiple User-Agent patterns?
Some companies use different User-Agents for different purposes. For example, OpenAI uses GPTBot for training, ChatGPT-User for browsing, and OAI-SearchBot for search. AttractOS detects all variants.
Bot Traffic by AttractOS