Tools
Crawl accessAI SEO Site AnalysisOne report on how ready your site is for ChatGPT, Gemini and AI Overviews: 40 checks, a score and the fixes that matter.AI Visibility CheckerCan ChatGPT, Claude, Perplexity, Gemini and Copilot see your page? One check, every engine.AI Site AuditCrawl your whole site and find every page AI crawlers can’t read, cite or reach.Cloudflare AI Crawler AnalyticsConnect Cloudflare with a read-only token and see real GPTBot, ClaudeBot and PerplexityBot traffic.Sitemap Checker for AI CrawlersValidate your XML sitemap and find URLs that are broken, redirected, noindexed or blocked for AI bots.AI Crawler CheckerSee which of 21 AI bots your robots.txt allows or blocks, with the exact rule that decides.AI robots.txt GeneratorAllow AI search, block AI training. Build a robots.txt for 21 AI bots in a few clicks.AI Bot Log AnalyzerUpload server logs and see which AI bots crawl you, what they read and which ones are fake.AI Bot IP CheckerIs it really GPTBot? Verify any IP against official AI crawler IP lists and reverse DNS.
Content for AIAI Crawlability CheckerSee your page the way AI crawlers do: raw HTML, no JavaScript, converted to Markdown.AI Citation CheckerWill ChatGPT quote your page? Score every passage for how easy it is for AI to cite.Query Fan-out GeneratorAISee the hidden sub-queries AI Mode and ChatGPT run for a search, and which ones your page covers.Schema Markup CheckerFull structured data audit: what JSON-LD you have, what’s missing for this page type, what’s wrong, plus ready code.LLM Token CounterCount GPT tokens for any text or web page and see which context windows it fits.llms.txt GeneratorBuild a spec-compliant llms.txt from your sitemap in seconds, then edit it visually.llms.txt ValidatorCheck any llms.txt against the spec: structure, links, duplicates and common mistakes.
Agents & trustPrompt Injection ScannerFind hidden instructions for AI on any web page: invisible text, comments, Unicode tricks, bot-only content.AI Agent Readiness CheckerCan AI agents find, understand and use your site? Checks MCP, A2A, OpenAPI, llms.txt and page usability.AI Entity CheckerDoes AI know your brand? Check Wikidata, Wikipedia and your Organization schema for consistency.AI Referral Traffic TrackerTrack visits from ChatGPT, Perplexity, Gemini and Claude in GA4 with a ready regex and channel setup.
LearnBlogAI trackerPricing
EN
EnglishENУкраїнськаUADeutschDEFrançaisFR
AI crawlersBotReady team

Cloudflare Pay Per Crawl in 2026: still in beta

Charging AI crawlers per request is still an experiment. Cloudflare keeps adding pieces, but most sites can’t switch it on yet.

Cloudflare launched Pay Per Crawl in 2025 to let sites charge AI crawlers for each request. In late 2026 Cloudflare’s documentation still describes it as closed beta.

What happened

The 2026 changelog shows steady additions:

  • January 13: a discovery API that lets verified crawlers find which domains take part, custom pricing, and a crawler error header with specific error codes.
  • April 17: an x402 payment-gated proxy Worker template for charging crawlers.

The documented minimum price is $0.001 per crawl. Stack Overflow has written about joining the programme. Some blogs report that Cloudflare is moving toward charging per citation or per use instead of per crawl. We couldn’t find that on Cloudflare’s own pages, so treat it as unconfirmed.

What it means for you

  • Most sites choose between allow and block in AI Crawl Control today. Charging is limited to beta participants.
  • Know your crawler traffic first. Pricing only makes sense if you know which bots visit, how often and which pages they take.
  • Per-crawler blocks override robots.txt. A block in AI Crawl Control returns 403 even if robots.txt allows the bot.

Our Cloudflare AI Crawler Analytics uses a read-only token to show AI crawler requests from your zone by bot and day.

Sources

  1. Cloudflare AI Crawl Control changelog
  2. Cloudflare docs, set a pay per crawl price
  3. Stack Overflow blog

Check your site

Cloudflare AI Crawler AnalyticsConnect Cloudflare with a read-only token and see real GPTBot, ClaudeBot and PerplexityBot traffic.AI Crawler CheckerSee which of 21 AI bots your robots.txt allows or blocks, with the exact rule that decides.