Making Your Website Agent-Ready: What Actually Matters in 2026

A growing share of the traffic that decides whether you get found is not a person. It is an AI agent or an answer engine — ChatGPT, Perplexity, Claude, Google’s AI Overviews — reading on someone’s behalf, choosing what to cite, and increasingly acting on what it finds.

Most websites are close to invisible to these systems. Not because the content is weak, but because it is wrapped in markup no model was built to read cheaply, with none of the signals they use to decide who to trust.

We rebuilt our own site to be legible to agents, and we run a free audit that scores anyone else’s. This is the honest field guide: what actually matters, in order, and what is hype dressed up as a checklist item.

Agent-readiness is not SEO — but they pull the same direction

Classic SEO is about ranking in a list of blue links. Agent-readiness is about whether an AI system can read, trust, and act on your site at all — before any ranking question comes up. A page can rank well and still be invisible to an answer engine, because the model could not cheaply extract a clean answer or resolve who you are.

The good news: almost everything you do for one helps the other. Structured data, a sitemap, clean metadata, a curated content map — all neutral-to-positive for classic search. You are not choosing between the two. You are covering the half of the job most sites skip.

What actually matters, in order

1. Structured data, so engines resolve you as a real entity

The single most under-used lever. schema.org JSON-LD — one linked @graph with your Organization, WebSite, and Service nodes cross-referenced by @id — tells an engine who you are and how your pages relate, instead of forcing it to guess on every page. Answer engines lean heavily on entity resolution to decide whether “SetKernel” is a real, citable thing. If your only signal is prose, you are asking the model to infer what a machine-readable block could state.

2. Markdown for agents — the token-economics win

This is the most under-appreciated one, and it is measurable. When an agent requests a page, it pays in tokens to parse your HTML — the <div> wrappers, the inline styles, the navigation it does not care about. Serve a markdown version on request (Accept: text/markdown) and the model spends its budget on your content instead. In one Cloudflare benchmark, a blog post dropped from roughly 16,000 tokens of HTML to about 3,000 of markdown — an ~80% reduction. Cheaper to ingest means likelier to be read in full, and likelier to be cited.

3. An AI-crawler policy — state how your content may be used

A robots.txt with Content-Signals, and an ai.txt, let you say in machine-readable terms whether AI systems may index, ground on, or train from your content. Silence is a decision made for you. If you want to be cited — most businesses do — say so explicitly, and name the crawlers.

4. llms.txt — hygiene, not a growth lever (the honest part)

llms.txt is a curated, machine-readable map of your site. It is worth having, and we ship one. But be clear-eyed: early large-scale measurements in 2026 show negligible bot traffic to llms.txt and no measurable citation lift. Treat it as hygiene, not a growth engine. Anyone selling it as the thing that will get you into AI answers is overselling a file that agents mostly are not fetching yet. Put the marginal effort into structured data and markdown instead.

5. An MCP server — the frontier where agents act, not just read

Everything above makes your site readable. A Model Context Protocol server makes it callable — an agent in Claude, Cursor, or ChatGPT can search your data, run a calculation, or start an enquiry as a tool. Almost no one has this yet, which is exactly why it is a differentiator rather than a checkbox. It is also the only item on this list that turns your site from a document an agent reads into a system an agent uses. (We wrote a separate guide on when an MCP server is worth building, and when it is overkill.)

What we did to our own site

We do not like selling a capability we have not run for ourselves, so we shipped all of it:

• An llms.txt and llms-full.txt, generated from a single source so the facts never drift.
• A robots.txt with Content-Signals and an explicit welcomed-crawler list.
• schema.org JSON-LD on every page, as one linked entity graph.
• Site-wide markdown negotiation, built as content negotiation on the Cloudflare Worker itself.
• A public, unauthenticated MCP server at /mcp — you can install it in Claude Desktop or Cursor right now — plus its discovery card and an API catalog.

We build the same for clients, and we published the MCP server template we start from, MIT-licensed.

What is overrated

• FAQ rich results. Google stopped showing FAQ rich snippets in May 2026. Keep the FAQ markup — AI systems still read it — but do not expect the SERP dropdown.
• Keyword density. Answer engines reward a clean, direct answer near the top of the page far more than a keyword hit count.
• Chasing every emerging spec. There is a new draft standard every month — DNS-based discovery, browser-side agent APIs, and more. Some will matter; most have no agents consuming them yet. Ship the standards that are stable and being read, and revisit the drafts when something actually reads them.

How to check where you stand

We built a free AI-readiness audit that scores any public site on exactly these surfaces — llms.txt, structured data, AI-crawler policy, markdown negotiation, an MCP server, and the basics — and emails you the checklist. It is the same tool we would run on you in a first conversation. Run it on your own site, and on ours.

The shift is already underway

The question is moving from “how do I rank” to “am I in the answer”. A site that is a wall of unlabelled markup is not wrong — it is just not in the answer. Being agent-ready is how you stay quotable as that shift accelerates, and most of it is a one-time engineering job.

If you want a hand with any of it, tell us what you are working on — two paragraphs is enough, and we reply in writing within one business day.