Skip to content
VectorCite
Free audit

Glossary entry claude-searchbot

Claude-SearchBot

Anthropic's retrieval crawler for Claude's web tool, separate from ClaudeBot (training).

Filed under
Engines & bots

Definition

Claude-SearchBot is Anthropic's retrieval-side crawler, the one that fetches pages when Claude's web-search tool is invoked. Like OpenAI's split, it's separate from ClaudeBot which is used for training. Blocking it kills Claude's citation eligibility.

Example

User-agent: Claude-SearchBot\nAllow: / (allows Claude to cite you when users ask questions)

How to optimize

Allow Claude-SearchBot in robots.txt. Anthropic also fetches /llms.txt, so publish one. VectorCite's AI-bot access signal flags this automatically.

Check a page against a real AI answer

Claude’s live answer to your buyer’s question, its sources, your readiness score and the fixes. Free, no signup, about half a minute.

Run a free audit