Technical GEO

Trakkr scan (833,791 domains): 7,029 sites embed 'remember/cite us' instructions in AI buttons. 37% use memory-anchoring language; 98% target ChatGPT. Microsoft calls it 'AI Recommendation Poisoning'.

OtterlyAI study of 1M+ citations found 73% of sites block AI crawlers via robots.txt, CDN rules, or JS rendering issues. Fix crawler access before anything else.

SearchScore SAVI (850k sites, Q2 2026): 74.2% Invisible/Low visibility; AI visibility averages 34.1 vs technical health 70.1, with structured data scoring 23.1.

Autonomous AI agents from Claude and GPT-5 now execute multi-step purchasing and comparison tasks. Agent-first content structure becomes critical for visibility.

OpenAI Agentic Commerce Protocol (ACP, with Stripe), Web Capabilities Protocol (WebMCP, Google+Microsoft), Universal Commerce Protocol (UCP, Google+Shopify) — sites without machine-readable

BrightEdge data shows AI agent requests now rival human search traffic, with OpenAI agents dominating. Differentiated robots.txt strategy is essential.

Presenc AI 2022-2026 trend: GPTBot grew 305% YoY. AI bots now 22% of all bot traffic. Meta-ExternalAgent surged to #2 at 16.7%. Bots = 31.2% of all HTTP requests; trajectory crosses human traffic

OpenAI runs 3 bots: GPTBot (training), OAI-SearchBot (search index), ChatGPT-User (fetches). Block OAI-SearchBot = zero ChatGPT citations. Most teams conflate them.

Known Agents data shows AI search crawlers are a measurable and growing portion of web traffic. Differentiated robots.txt management is a basic GEO hygiene requirement.

JetOctopus server log analysis (Feb 2026): AI bots ~40-50% of Googlebot-level activity. 65-70% of AI bot traffic is user search, not training. 14+ AI user-agents need explicit Allow rules in

Trakkr (575,788 AI crawler visits): GPTBot reaches homepages only ~3% of the time; 88.5% of pages get exactly one visit; 21% of ChatGPT Search sessions start on blog pages.

Trakkr cross-referenced 337K AI citations with 11.4M crawler visits: what AI crawls (product pages) is not what it cites (blog content). Measure both, not one.

HTTP Archive: chatgpt.com, claude.ai, gemini.google.com lack llms.txt. But docs.anthropic.com, docs.perplexity.ai have one. Value is for agent-readiness, not marketing.

Google merged AI Overviews and AI Mode into one unified AI Search experience at I/O 2026. Runs on Gemini 3.5 Flash. Single citation pool eliminates need for separate strategies.

Google merged AI Overviews and AI Mode into one unified AI Search experience at I/O 2026. Runs on Gemini 3.5 Flash. Single citation pool eliminates need for separate strategies.

ClaudeBot (training), Claude-Web (real-time browsing), and anthropic-ai (catch-all) can be independently controlled in robots.txt for granular AI visibility management.

ChatGPT Search retrieves from Bing's index: 87% of ChatGPT cited pages correspond to Bing top results (Mersel AI 2026). Submit sitemap to BWT, allow OAI-SearchBot, enable IndexNow.

AirOps study of 548K pages: ChatGPT Search retrieves via Bing index, checks OAI-SearchBot crawl access, then cites only 15% based on structure, freshness, and authority.

Previsible study: ChatGPT sends 28.8% of its referrals to internal search results pages. The model trusts the domain but defaults to site search when it can't identify the right page.

Profound (201K prompts, 812K product cards): card info is pulled from the rank-1 offer; GPT-tagged products get +144% top-rank lift; median review count +123%; Reddit = ~1/3 of shopping citations.

ChatGPT uses Bing to retrieve candidates, then a fine-tuned model re-ranks by answer fit, domain authority, source consensus; zero JS execution.

Profound 2025 + multiple 2026 studies: Claude's web retrieval runs on Brave Search's independent index. Google rankings have limited transferability. Brave does not license from Google or Bing.

Presenc AI study of top 1,000 sites: ClaudeBot crawls 20,583 pages per referral. PerplexityBot best at ~210:1. 25% block GPTBot. OAI-SearchBot at 85:1.

Cloudflare CDN-level bot blocking overrides robots.txt allow rules, an invisible cause of ChatGPT invisibility even when Bing-ranked.

New robots.txt extension declares post-fetch usage (search/ai-input/ai-train). Set ai-train=no while ai-input=yes to stay citable but not trained on.

From Sep 15, 2026, Cloudflare blocks multi-purpose crawlers by default on ad-supported pages. Publishers can granularly allow/block each tier.

Presenc AI (April 2026) joins crawl events to citation outcomes. PerplexityBot 5-10x more efficient than GPTBot. Optimize differently per engine — Perplexity for fetchable content, Claude for

Seller-controlled GEO rewrites promote flawed products into LLM recommendation sets by up to 83.2%; structured evidence checks cut the harm by up to 39.2% (SafeGEO, 600 cases).

Google launched always-on AI Mode information agents on June 12 2026 for Ultra subscribers. Unlike zero-click AIO, they push source-linked updates proactively to users.

Google rolled out hover pop-up link cards in AI Overviews and AI Mode: hovering over highlighted text shows link cards for direct source access, creating a new path for user clicks from AI summaries.

Google launched AI image generation directly within AI Overviews (July 2026) using the Nano Banana model. Rolling out in English for AI Mode-supported countries.

Google announced AI Mode as default replacement for traditional search, Gemini 3.5 Flash as default model, information agents, and Personal Intelligence across 98 languages. AI Mode crossed 1B MAU.

Google merged AI Overviews and AI Mode into one unified AI Search layer on Gemini 3.5 Flash, reaching 1B+ monthly users, eliminating separate citation strategies.

Chrome Lighthouse 13.3 (May 2026) added an Agentic Browsing audit category testing llms.txt presence, WebMCP support, accessibility-tree quality and CLS — an agent-era readiness signal.

Google confirmed in 2026 that no AI-specific markup or llms.txt is required for AIO or AI Mode: 'GEO is still SEO at the core.'

Google's S-CTS terminates entire AI content networks (not individual pages), while S-BERT detects the mathematical 'fingerprint' of AI text. New model outputs detectable within days via LoRA updates.

Google's SAGE paper (Jan 2026) reveals AI agents perform multi-step searches. Top-3 rankings are the gateway to agentic discovery. Four 'shortcut' patterns identified.

TUM thesis study of 370K+ results found ChatGPT Business citations are 95% Bing-sourced, Plus citations 94% Bright Data (Google scraper).

FuelOnline analysis: Grok DeepSearch draws on Bing's index. Submitting via IndexNow immediately after publishing is the fastest path to Grok citation visibility.

ProAISearch found Grok's crawler (user agents 'xAI' and 'Grok') lacks 'bot' in its name, so most default security setups and WAF rules silently block it. Content never gets indexed.

Otterly's 3-month Copilot data: 647 grounding queries, 30,398 grounding events; 5 pages carried 74.6% of citations, 99.6% of AI influence invisible. Fix high-grounding/low-citation pages.

HasData analysis of 10,894 domains (July 2026): 56.4% of news publishers block at least one AI crawler. GPTBot banned by 50.5% of publishers. 39.5% of GPTBot blocks fail to actually block.

Google's April 2026 web sweep found attackers seeding prompt injections on sites to corrupt browsing AI; data-layer governance now required.

Erlin's 2026 data: AI parse success rates — static HTML with schema 94%, plain HTML 68%, JS-rendered 23%, PDF 7%. JS-rendered pricing/feature pages often read as empty templates.

Google's June 24-26 2026 Spam Update extended SpamBrain enforcement to AI Overviews and AI Mode. Tactics to game AI answers are now treated as spam equivalent to paid links.

Presenc AI State of llms.txt 2026: 784+ implementations but only Perplexity and Anthropic (Claude Desktop) confirmed use. OpenAI, Google, Meta silent. MCP emerging as alternative.

Presenc AI April 2026 report: Anthropic and Perplexity confirmed llms.txt support. Adoption <10% in finance/healthcare, high in developer SaaS. Top-100 brands lead.

llms.txt (proposed by Jeremy Howard Sep 2024) adopted by 4-5% of mid-market sites by mid-2026. Current evidence shows no direct citation boost but helps AI agents discover pages faster.

One-van firm scored ChatGPT 99/100 against national brands using 9 AI Discovery Files. Backlinks didn't determine AI assessment — structured facts did.

OtterlyAI's 90-day controlled experiment: out of 62.1K total AI bot hits, only 84 went to /llms.txt. Ahrefs: no major LLM provider currently supports llms.txt. Adoption is ~2% across 1M sites.

The Concurate case study documented a 5x increase in AI-referred traffic after implementing llms.txt. An llms.txt file at the domain root signals content priorities to LLM crawlers.

SE Ranking analyzed 300K domains: llms.txt has no effect on AI citations. Removing the variable improved model accuracy. 7.4% adoption in top 10K sites. Niche value for developer docs only.

Ahrefs studied 137,000 domains: ~28% publish llms.txt but 97% of those files got zero fetches in May 2026. Google confirms it ignores the file; coding agents fetch it ~5x more than AI search bots.

llms.txt (proposed by Jeremy Howard Sep 2024) adopted by 4-5% of mid-market sites by mid-2026. Current evidence shows no direct citation boost but helps AI agents discover pages faster.

SE Ranking analyzed 300K domains: llms.txt has no effect on AI citations. Removing the variable improved model accuracy. 7.4% adoption in top 10K sites. Niche value for developer docs only.

Botify's live-server experiment: bots discover Markdown via links, head tags, llms.txt, or content negotiation. GPTBot took 1,273 reads via links/head; purpose-built AI tools request Markdown ~100%

Foglift classified 65,527 AI crawler requests: 53.7% training, 32% indexing, 14.3% user-triggered fetch. Blocking training bots won't stop citation crawls.

Two converging findings: AI systems fetch pages with ~2s timeout (HTTP 499), and SE Ranking finds pages with FCP under 0.4s average 6.7 citations vs 2.1 for slower pages — a 3.2x gap.

Machine Relations Index measurement shows Perplexity's retrieval architecture systematically deprioritizes paywalled sources — affecting how B2B brands with gated content appear.

Perplexity's first technical explainer (Aug 6, 2026) details post-training models to connect evidence across sources and verify facts - shaping how content is synthesized and cited.

Perplexity's pipeline (BM25 + dense retriever + XGBoost reranker, 0.7 threshold) retrieves 5-10 pages and cites 3-4. Crawler access is a binary gate: robots.txt/Cloudflare blocks remove pages

Perplexity processes 780M+ monthly queries via 3-layer reranking: relevance 30%, visual placement 20%, domain authority 15%, freshness 15%, source diversity 10%, schema 10%.

Researchers discovered ChatGPT's VIP tier via network traffic analysis. Licensed publishers (Reuters, WSJ, Wikipedia) get pre-summarized full-article extracts.

Botify case study: Rail Europe shifted from volume-led to value-led SEO, reorganized sitemaps by page type, and prioritized AI crawler governance — resulting in 2x ChatGPT referral traffic.

OAI-SearchBot/PerplexityBot/ClaudeBot power citations; GPTBot/Google-Extended affect training only. Blocking the wrong one invisibilizes you.

Otterly AI analyzed 1,028,959 URLs cited across 6 AI platforms. URL length correlates near-zero with citations (r = -0.025). /guide/ pages average 2.7 citations (42% above 1.9 avg). Clean URLs beat qu

89% of AI crawler traffic is training/mixed, only 8% search-related. Allow OAI-SearchBot, PerplexityBot, Claude-Web. Block GPTBot, CCBot, Bytespider.

Vercel analysis of 1.3B fetches found near-zero JS execution for GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot; only Google-Extended renders JS.

Stripe's llms.txt includes 'Instructions for Large Language Model Agents' section. Explicitly steers coding assistants away from legacy Card Element. Real value of llms.txt is agentic-web control

Stripe's llms.txt includes 'Instructions for Large Language Model Agents' section. Explicitly steers coding assistants away from legacy Card Element. Real value of llms.txt is agentic-web control, not

Hedges Company case study: Product schema markup + llms.txt drove 200% monthly AI referral traffic growth for 3 months. ChatGPT/Perplexity citation rates tripled month-over-month.

Hedges Company case study: Product schema markup + llms.txt drove 200% monthly AI referral traffic growth for 3 months. ChatGPT/Perplexity citation rates tripled month-over-month.