By September 2026, at least 17 open protocols decide whether an AI agent can read, trust, operate, and buy from your website. Two years ago most of them did not exist. Now they stack into six layers, from "who may crawl you" to "let the agent check out with a stablecoin."

This is the map. Each protocol gets a plain explanation, who backs it, and one honest verdict: act now, worth adding, or watch. GenReady's analyzer checks the starred (★) items and scores your site against them.

Protocol statuses re-verified September 2, 2026. Eight entries moved since the July edition. The biggest: WebMCP is now switched on by default across Shopify and Cloudflare, and Web Bot Auth got its first IETF working group draft on September 1. This page updates as the specs move.

The whole stack at a glance

The stack answers six questions in order. Who may use your content? Can you trust the agent? What do you offer? Can the agent read it cheaply? Can it operate your site? Can it pay?

Layer 1: Who is allowed to crawl and use your content?

This layer sets the terms before an agent reads a single word. It covers permission and price.

robots.txt now carries AI-specific rules, and Cloudflare's Content Signals extend it. Content Signals let you say "allow search, block AI training" in a machine-readable line. September 15, 2026 is still the switch date, and the defaults are unchanged: every new domain onboarding to Cloudflare gets search allowed, training and agent use blocked on pages that display ads. Cloudflare has since added a fourth field, use=, with three values: immediate (interact, store nothing), reference (index and link back, the default), and full (summarize and reproduce).

One caveat worth your time. On July 6, 2026, Google's John Mueller said Content Signals has "no effects whatsoever for any crawler or llm" and that Google has no plan to support it. So this is a declaration of terms, not an enforcement mechanism. Set your AI directives anyway. They cost nothing and they establish intent.

RSL (Really Simple Licensing) turns robots.txt into a licensing contract. It adds machine-readable terms for attribution, pay-per-crawl, and pay-per-inference. It launched in September 2025 with Reddit, Yahoo, Medium, Quora, and Ziff Davis. Roughly 1,500 media organizations now endorse it, including the Associated Press and People Inc. Worth adding if your content has licensing value.

Cloudflare Pay Per Use is the enforcement cousin of RSL. Announced July 1, 2026, it evolved from Pay-Per-Crawl. Publishers get paid when content creates value, not just when a bot fetches it. Two months on it remains early, with Ceramic.ai and You.com as the launch AI partners. It needs Cloudflare's CDN and actively blocks non-payers, where RSL only states terms. Watch for now unless you are already deep in Cloudflare.

Layer 2: How does an agent prove who it is?

Right now most agents announce themselves with a user-agent string, and anyone can fake that. Layer 2 fixes the spoofing problem with cryptography.

Web Bot Auth crossed a real line on September 1, 2026. The IETF published draft-ietf-webbotauth-httpsig-protocol-00, the working group's first adopted document, on the standards track. The older architecture draft that carried this work, draft-meunier-web-bot-auth-architecture-05, has expired and been replaced.

The mechanism has not changed. Agents sign requests with Ed25519 HTTP Message Signatures (RFC 9421) and publish public keys at /.well-known/http-message-signatures-directory. Claude, ChatGPT, Perplexity, and Common Crawl already sign. Cloudflare, AWS WAF, Vercel, Shopify, and Akamai already verify.

The schedule slipped, though. The group targeted standards-track documents at the IESG in April and a BCP by August 2026. Neither landed on time, and the BCP is still stuck on whether its candidate draft fits the charter. Watch: verification is handled by your CDN or host, not by you directly.

Layer 3: How do agents discover what your site offers?

Layer 3 is the signposting layer. It uses well-known files to tell an agent what lives on your site before it crawls blindly. GenReady checks most of these.

★ MCP Server Card lives at /.well-known/mcp/server-card.json. If you run a Model Context Protocol server, the card lets clients like Claude find and connect to it automatically. No card means clients must be configured by hand.

MCP shipped its largest revision on July 28, 2026. The core went stateless: the initialize handshake and the Mcp-Session-Id header are gone, and method names now travel in HTTP headers so gateways can route without parsing JSON. Roots, sampling, and logging are deprecated with a twelve-month runway. If you run a server, that migration is your September work. Act now on the card once the server is current.

★ A2A Agent Card lives at /.well-known/agent-card.json and describes your agent to other agents. A2A reached v1.0 in April 2026 with signed cards and over 150 organizations at the Linux Foundation's one-year mark. No newer release has shipped since. It is generally available in Microsoft Copilot Studio, Azure AI Foundry, and Amazon Bedrock AgentCore. Act now if you expose an agent.

llms.txt is a markdown index of your key pages, at your site root or covering any path beneath it. SE Ranking crawled about 300,000 domains and found the file on 10.13% of them. The same study found no measurable citation effect. Statistical tests showed no correlation, and dropping llms.txt from their model made it more accurate. Search Engine Land's smaller check of nine sites saw no lift on eight. No study since has contradicted either.

The spec reached v2 on August 10, 2026, and the revision is worth knowing about. It added two link relations so agents stop guessing where things are, rel="describedby" for the llms.txt covering a page and rel="alternate" type="text/markdown" for that page's markdown version, both settable as an HTTP header.

It also says outright that your llms.txt links should point at markdown versions rather than HTML. That makes it the index layer over your markdown negotiation rather than a competitor to it. OpenAI, Anthropic, and Google publish one for their own developer docs, and Lighthouse now audits that the file fetches cleanly. Worth adding because it is cheap and because v2 gave it a job, but the citation evidence has not moved.

agents.json proposes a way to advertise your API actions to agents. It sits at v0.1.0, comes from one startup rather than a standards body, and adoption stays niche. Watch it; skip it for now.

Layer 4: What formats can agents read cheaply?

Agents pay by the token. HTML is expensive and noisy. Layer 4 is where the real, boring wins live.

★ Markdown content negotiation is the sleeper of the whole stack. An agent sends Accept: text/markdown, and your server returns clean markdown instead of bloated HTML. That cuts 70-90% of the tokens.

The sender list has doubled since the February 2026 Vercel and Checkly data. Seven clients now send the header: Claude Code, Cursor, OpenCode, Copilot Chat, Copilot CLI, Microsoft Copilot, and OpenClaw. Codex CLI takes the v2 route and follows rel="alternate" instead.

Read the other column too. ChatGPT browsing, Claude.ai, Perplexity, Gemini, and Grok all still fetch plain HTML, and Googlebot never asks. This is a coding-agent convention so far, not a search-engine one. Act now anyway: serving markdown is the cheapest win on this page, and since v2 it is also what your llms.txt links should point at.

★ schema.org is the veteran of the stack and still the highest-impact move. Structured data tells engines exactly what your page is: a product, a recipe, an FAQ, a review. AI answer engines lean on it heavily to cite sources, and nothing about that changed this quarter. Act now, and see GenReady's schema markup guide for the full playbook.

Layer 5: How do agents operate your site?

Discovery tells agents what exists. Layer 5 lets them do things: search, filter, add to cart, submit. This is where the stack moved most since July.

★ WebMCP exposes your site's functions as callable tools through navigator.modelContext. In July it was an origin trial nobody had turned on. In August it became the default for a large slice of the commerce web, twice in two days.

On August 5, 2026, Shopify switched WebMCP on for every Liquid storefront, with Hydrogen in developer preview. Ten tools ship out of the box, covering catalog search, cart, checkout, orders, and policy lookup. No app, no admin setting, no merchant asked. On August 6, Cloudflare shipped the same idea for any site behind its network, as one dashboard toggle with nothing to deploy.

Two limits keep this honest. The spec is still a W3C Web Machine Learning Community Group draft, last revised July 28, 2026, and not on the standards track. And it only runs in Chromium, through the Chrome 149 to 156 origin trial. Firefox and Safari sit in the group and have shipped nothing.

The verdict flips anyway. If you sell on Shopify or sit behind Cloudflare, WebMCP is not something to watch. It is already exposing tools on your site, and you should go read what it hands an agent. Full breakdown: should your website implement WebMCP now.

★ NLWeb, from Microsoft at Build 2025, gives your site an /ask endpoint. It turns natural-language questions into schema.org JSON, plus an /mcp endpoint. Early adopters include Shopify, TripAdvisor, Eventbrite, O'Reilly, and Hearst. Build 2026 in June gave it stage time and no adoption jump. The list is the same list. Honest verdict: watch, do not build yet unless you have a strong reason.

★ Agent Skills package reusable instructions an agent can load, listed at /.well-known/skills/index.json. Anthropic launched them in October 2025 and opened the standard at agentskills.io on December 18, 2025. The compatible-client list now runs to 46 products, up from about 40 in June. ChatGPT and Codex, Gemini CLI, GitHub Copilot, Cursor, Mistral Vibe, Databricks, and Snowflake all read the format. That is every major lab's coding agent on one spec. Worth adding if you want agents to use your product well.

Layer 6: How do agents pay?

The top of the stack is money. Four protocols now let an agent complete a purchase, and they split into commerce rails and payment rails.

★ UCP (Universal Commerce Protocol) is the commerce front-runner, and it is compounding. Google and Shopify announced it January 11, 2026 at NRF with Etsy, Wayfair, Target, and Walmart. On April 24, Amazon, Meta, Microsoft, Salesforce, and Stripe joined the Tech Council, taking it to ten of a possible sixteen seats. Verified stores went from 8,000 in mid-June to 16,171 on August 31, a double in ten weeks, still overwhelmingly Shopify, where it ships by default. It is live in Google AI Mode, Gemini, and YouTube Shopping, with a manifest at /.well-known/ucp. Act now if you sell online, and read our full UCP guide.

ACP (Agentic Commerce Protocol) is UCP's rival, from OpenAI and Stripe. It has been live in ChatGPT since September 2025 and is still in beta, on an April 17, 2026 stable snapshot. All-in fees run near 7.2%, versus roughly 3.2% for UCP, because ACP stacks a platform commission on top of card processing. Merchants who support both report up to 40% more agentic traffic. If you sell online, dual-protocol is the safe bet.

★ x402 is the payment rail: micropayments over the dormant HTTP 402 status code, settled in stablecoins. Coinbase open-sourced it in May 2025. The x402 Foundation reached operational launch under the Linux Foundation in July 2026 with 40 members, and the premier tier now reads like the payments industry itself: Visa, Mastercard, Amex, Stripe, Adyen, Fiserv, Ripple, Circle, Google, Shopify, AWS, and Coinbase. AWS CloudFront support went GA in July, and Cloudflare's Monetization Gateway launched July 1 and now charges for any resource behind Cloudflare.

Now the part the press releases skip. Transfer counts are enormous and the dollars behind them are not. The week of August 17, 2026 was the busiest of the year at 8.7 million transfers, and those transfers moved about $368,000 between them. Settlement volume is down roughly 93% since January, according to analyst Jamie Coutts, against daily peaks near $800,000 in late 2025.

A population-scale study published this summer explains why the counts look healthier than the business. Researchers classified 136.7 million Base settlements from September 2025 through June 2026 and found 21% provably self-dealing, another 64% moving inside funding-linked operator clusters, and 15% reaching anyone outside them. Of 25,163 services advertised on Base, 624 ever received a payment and 249 ever earned $10. Settlement count, the authors point out, is the one number an interested party can manufacture for free.

The rails are real. The economy on top of them is still mostly testing itself. Watch, and pilot if you sell APIs or per-call access.

AP2 (Agent Payments Protocol), from Google on September 16, 2025, coordinates the authorization behind a purchase. Version 0.2.0 in April 2026 added "Human Not Present" autonomous purchases and Verifiable Intent, a tamper-proof log of what the user actually authorized, built with Mastercard.

The donation went through. On April 28, 2026, Google handed AP2 to the FIDO Alliance and Mastercard handed over Verifiable Intent. FIDO stood up an Agentic Authentication Technical Working Group, joined by Visa, Amex, PayPal, Stripe, Adyen, Cloudflare, Microsoft, and Okta. Governance is settled. Deployment is not: Google is still the only one shipping it. Watch.

What is on the horizon?

Three efforts aim higher than any single site. AGNTCY, backed by Cisco and LangChain and now past 65 member companies at the Linux Foundation, builds an open agent schema and has wired its Directory to the NANDA Index so agents in different registries can find each other. DNS-AID, contributed by Infoblox in May 2026, does agent and MCP server discovery over plain DNS.

AgentReady is the odd one out. Released at v1.0 in August 2026 by ora.ai and Vercel, it is not a protocol. It is a community spec of which protocols a site should implement, organized around what agents do: find, read, act. If this article is the map, AgentReady is a second opinion on it. None of the three needs action from a website owner today. Track them; do not build for them yet.

Where should a typical site owner start?

Do not chase all 17. Most sites get 90% of the benefit from four moves, in this order.

1. Add schema.org markup and serve markdown to agents. Both are cheap, proven, and cut the cost of citing you.

2. Publish agent or server cards if you run an API, an MCP server, or an agent. If you run a server, migrate it to the July 28 stateless spec first.

3. Turn on UCP (and ideally ACP too) if you sell online.

4. Audit what WebMCP already exposes if you are on Shopify or Cloudflare. This is no longer a watch item for those sites.

That sequence maps to the layers: content first, discovery second, transactions third, interaction last.

What this means for your website

The web spent 30 years optimizing for two visitors: humans and search crawlers. A third visitor now arrives in volume, and it reads, trusts, operates, and pays differently. The 17 protocols above are how that visitor decides whether your site is usable or invisible.

The good news: you do not need all of them. You need the right few, in the right order, and you need to keep watching the rest. Most of this stack is under two years old and moving fast. WebMCP proved the point this summer: a watch-list item became a default on hundreds of thousands of storefronts in a single week, and nobody sent the merchants a memo.

Start with the boring wins. Schema and markdown pay off immediately. Layer on discovery and payments where they fit your business. Revisit this map often.

Want to know which of these protocols your site already passes? GenReady's analyzer checks the starred items and scores your agent readiness in under 60 seconds at genready.ai.