Ready for AI agents
An AI that clicks for you
An answer engine reads pages and writes an answer. An AI agent goes further: it opens websites, clicks links, fills in forms and compares options to finish a job for a person, like "find three space shooters that work offline and put the best one in my basket". Claude can do this in the Chrome browser, and other AI companies make agents too.
Agents want plain text instead of fancy layouts, files at standard addresses, and clear rules about what they may do. A site that offers those is agent-ready.
Is It Agent Ready?
isitagentready.com is a free scanner made by Cloudflare, launched in April 2026. Type in a website address and it fetches the home page plus a set of well-known addresses, standard places like /robots.txt or /.well-known/... where agents expect to find files.
Its own guide counts 22 checks in five categories: Discoverability, Content Accessibility, Bot Access Control, API / Auth / MCP, and Commerce. Each check shows pass or fail with evidence, plus fix instructions you can paste into a coding agent. Cloudflare modelled it on Google's Lighthouse, which scores pages for speed and best practices.
Six levels, 0 to 5
The scan gives one overall level. Each level needs everything below it:
- 0 Not Ready: fewer than two of the level 1 basics.
- 1 Basic Web Presence: two of robots.txt, sitemap and Link headers.
- 2 Bot-Aware: plus AI bot rules and Content Signals.
- 3 Agent-Readable: plus Markdown negotiation.
- 4 Agent-Integrated: plus one of MCP Server Card, A2A Agent Card, Agent Skills or API Catalog.
- 5 Agent-Native: plus two of Web Bot Auth, all of those integrations, or sign-in metadata.
Commerce checks are reported but don't count toward the level yet.
Quick check
The Hyper Shards site has a robots.txt and a sitemap, but robots.txt has no AI bot rules or Content Signals. Which level does it get?
- Level 1, Basic Web Presence
- Level 2, Bot-Aware
- Level 0, Not Ready
- Level 5, Agent-Native
Show the answer
Level 1, Basic Web Presence
Two of the three basics (robots.txt, sitemap, Link headers) reach level 1. Level 2 also needs AI bot rules and Content Signals in robots.txt.
Discoverability: can agents find things?
- robots.txtA valid crawl-rules file at /robots.txt, the first place a well-behaved bot looks. We pass with a small text file in our site folder.
- SitemapA sitemap.xml listing our pages, also named in robots.txt with a "Sitemap:" line so agents don't have to guess (the Search Console lesson covers sitemaps).
- Link headersLines in the home page's HTTP response that point to machine-readable files, using names like service-doc or describedby. On Cloudflare Pages, a _headers file adds them.
- DNS-AIDDNS for AI Discovery: DNS records under an _agents name that point to an agent the site runs. Hyper Shards has no agent of its own, so this can wait.
The first category has four checks. Here is what each one looks for, and how our studio site handles it.
Content and bot rules
- Markdown negotiationWhen an agent asks for a page with the header Accept: text/markdown, the site answers in Markdown, plain text with simple marks. Cloudflare measured up to 80% fewer tokens in some cases. Its Markdown for Agents switch does this on Pro plans and up, at the time of writing.
- AI bot rulesrobots.txt groups that name AI crawlers, like OAI-SearchBot or ClaudeBot. The AEO lesson explains search crawlers versus training crawlers.
- Content SignalsA robots.txt line that says how content may be used: search, ai-input (using pages in AI answers) and ai-train (training models).
- Web Bot AuthFor sites that send their own bots: publish public keys and sign each request, so others can check it came from you. It only counts toward level 5, and Hyper Shards sends no bots.
The next two categories cover how agents read our pages and what they may do with them. A token is a small chunk of text an AI reads, and every token costs time and money.
Read the signal
1User-agent: *2Content-Signal: search=yes, ai-input=yes, ai-train=no3Allow: /4 5Sitemap: https://ourstudio.com/sitemap.xml
What does line 2 ask AI companies to do with our pages?
- Use them for search and to answer people's questions, but not to train AI models
- Never show them in any search engine or assistant
- Pay the studio before reading any page
- Train models on them, but never link back to us
Show the answer
Use them for search and to answer people's questions, but not to train AI models
search=yes allows search, ai-input=yes allows using pages in AI answers, and ai-train=no asks companies not to train on them. Like the rest of robots.txt, it states our wishes. It is not a lock against badly behaved bots.
API, Auth and MCP: can agents use our tools?
- MCP Server Card, WebMCPA card at /.well-known/mcp/server-card.json describes an MCP server before an agent connects. WebMCP lets a web page offer tools, like "search levels", to an agent in the browser.
- Agent Skills, A2A Agent CardAn index of instruction files agents can download, at /.well-known/agent-skills/index.json. An A2A Agent Card describes the site's own agent to other agents.
- API Catalog, ARD manifestA list of the site's APIs and their docs at /.well-known/api-catalog. An ARD manifest lists every agent tool in one file.
- OAuth discovery, OAuth Protected Resource, Auth.mdFiles that tell an agent how to sign in with permission, using OAuth, instead of asking a person for their password.
Nine checks are for sites that offer tools or accounts, not just pages. Many use MCP (Model Context Protocol), an open standard for connecting AI apps to tools. Most checks look for a JSON file at a well-known address.
Our static game site has no API or player accounts for agents yet, so these can wait. A first step toward level 4 could be an Agent Skills index.
Read the scan
1Level 1: Basic Web Presence2 3robots.txt pass4Sitemap pass5Link headers fail6AI bot rules fail7Content Signals fail8Markdown negotiation fail9MCP Server Card fail
This is a scan of the Hyper Shards site. Which fix takes it to level 2, Bot-Aware?
- Add AI bot rules and a Content-Signal line to robots.txt
- Build an MCP server with a server card
- Turn on Markdown negotiation, and nothing else
- Add Link headers, and nothing else
Show the answer
Add AI bot rules and a Content-Signal line to robots.txt
Level 2 needs level 1 plus both AI bot rules and Content Signals, and both live in robots.txt, so it's a small text edit. Markdown negotiation is the level 3 step, and an MCP Server Card belongs to level 4.
Commerce, and who stays in charge
The last category looks for new ways agents can pay: x402 (built on the web's 402 "Payment Required" status code), MPP (Machine Payment Protocol), UCP (Universal Commerce Protocol) and ACP (Agentic Commerce Protocol). These checks don't count toward the level yet, and the standards are young.
Hyper Shards takes payments through Stripe, on an account a parent owns. Letting agents buy for people is a business decision for a parent, not a quick setting. A score is a guide, not a grade: fix the easy, useful checks first.
Bolt says: The scanner's own advice: start with robots.txt, AI bot rules and a sitemap. Then take on the stage boss!
Fact-checked 9 October 2026
- Is It Agent Ready? isitagentready.com
- Cloudflare blog: agent readiness blog.cloudflare.com
- Cloudflare: Markdown for Agents developers.cloudflare.com
- Cloudflare Pages: headers developers.cloudflare.com