Skip to content
VectorCite
Free audit

How each AI engine finds and cites sources

One guide per engine: how it retrieves pages, which signals it leans on, the crawlers to allow in robots.txt, and what to change for it specifically.

What VectorCite checks

Claude, ChatGPT
In the free audit with web search, each asked several times
Gemini
In the free audit from memory; its search answers aren't measured (Google's terms)
Perplexity
In the console where its key is set, or with your own key
AI Overviews, Copilot
In the console where their keys are set
AI Mode, You.com
Explained here, not measured

Pick an engine

8 guides. Each one covers retrieval, the crawlers it identifies as, the signals it weights and how to optimize for it.

  • OpenAI

    ChatGPT Search

    OpenAI's web-retrieval-enabled chat surface, the largest AEO target by user volume.

    Crawlers
    GPTBot, OAI-SearchBot, ChatGPT-User
    In VectorCite
    In the free audit

  • Anthropic

    Claude (with web tool)

    Anthropic's frontier model with its web_search tool. It values structural depth + provenance.

    Crawlers
    ClaudeBot, Claude-Web, anthropic-ai
    In VectorCite
    Live in the free audit

  • Perplexity AI

    Perplexity

    Pure RAG answer engine: every response is grounded in cited sources.

    Crawlers
    PerplexityBot, Perplexity-User
    In VectorCite
    In the console, with a key

  • Google

    Google AI Overview

    Google's generated answer block above SERPs, the single highest-volume AEO surface.

    Crawlers
    Googlebot, Google-Extended (for training data only)
    In VectorCite
    In the console, with a key

  • Google

    Google AI Mode

    Google's full-page conversational answer surface, with deeper sub-query fanout than AI Overview.

    Crawlers
    Googlebot, Google-Extended
    In VectorCite
    Not measured

  • Google

    Gemini (Google)

    Google's frontier model with web search, distinct from AI Overview's SERP integration.

    Crawlers
    Googlebot, Google-Extended, GoogleOther
    In VectorCite
    From memory, in the free audit

  • Microsoft

    Microsoft Copilot (with Bing)

    The third-largest answer engine: Microsoft's chat surface using Bing's index.

    Crawlers
    Bingbot, MSNBot
    In VectorCite
    In the console, with a key

  • You.com

    You.com

    Independent answer engine with strong dev-tool integration and multi-model support.

    Crawlers
    YouBot, Genius (for code-search agent)
    In VectorCite
    Not measured

Are these crawlers getting in?

The robots.txt analyzer shows which AI bots your file lets in and which it blocks. The free audit then asks Claude your buyer’s question and shows whether your page is cited.

Check your robots.txtRun a free audit