How to make your website readable to AI agents

Most AI visibility advice skips the hard part

The advice is everywhere: add FAQ schema, publish an llms.txt, write answer-shaped content. All fine. All useless if the AI agent never got a readable page in the first place.

An AI assistant has to do four things before it can ever recommend you. Reach your site. Parse it. Understand it. Trust it. Fail any one and everything downstream is decoration.

So we did all four to our own site, in that order, and this is the whole build. Every claim below is fetchable on agentu.co.uk right now, which is rather the point of writing it.

Layer 1: Be reachable, or nothing else counts

Most AI agents do not run your JavaScript. Search crawlers have rendered it for years and people assume the AI fetchers do too. Largely they don't. They pull your HTML, read it as text, and move on inside a short timeout.

So if your headline, your prices, or your proof only exist after a script runs, they do not exist. Not to the agent, and not to the buyer who asked it.

agentu.co.uk is a static build served from Cloudflare's edge. No server-side rendering, no client-side framework, no hydration. You can turn JavaScript off entirely and still read every word of every page. The scripts we do ship are enhancements: the contact form, the diagnostic, a load-time readout, a table of contents. Delete every one of them and the site still says everything it says.

Then we get out of the agent's way:

• robots.txt allows everything and points at the sitemap. No AI-crawler blocklist. We want to be read.
• sitemap.xml lists every indexable page, and the build fails if the sitemap and the build output disagree. A sitemap that lies is worse than none.
• No cookie wall. No interstitial. No newsletter popup covering the content. Every one of those is a door an agent will not open.

Layer 2: Be parseable

Reachable gets you the bytes. Parseable is whether the bytes mean anything.

Semantic HTML, one <h1>, headings that match the questions people actually ask. Then the structure that answer engines lift:

• An answer block. Every article opens with a 40 to 80 word direct answer, written to survive being quoted on its own with our name attached. That is what a citation is.
• Key takeaways. Liftable, self-contained sentences near the top.
• A visible FAQ. Real questions, real answers, on the page.

One rule holds all of this together: write so that any single paragraph could be quoted alone and still be true, still be useful, and still name us.

Layer 3: Be understood, with one graph and one identity per thing

This is the layer almost everyone gets half right, and it is the one that compounds.

Most sites emit structured data as a pile of disconnected snippets: an Organization block here, a FAQ block there, a Person repeated inline on every article. Each one is individually valid. Together they say nothing, because nothing links to anything.

We emit one connected @graph per page. Every entity appears once, carries exactly one @id, and other nodes reference it by that @id rather than describing it again:

{
  "@context": "https://schema.org",
  "@graph": [
    { "@type": "Organization",
      "@id": "https://agentu.co.uk/#org",
      "name": "Agentu Ltd",
      "makesOffer": [{
        "@type": "Offer",
        "price": "795", "priceCurrency": "GBP",
        "priceSpecification": {
          "@type": "UnitPriceSpecification",
          "price": "795", "priceCurrency": "GBP", "unitText": "month" },
        "itemOffered": {
          "@type": "Service",
          "@id": "https://agentu.co.uk/signal/#service",
          "name": "Agentu Signal",
          "provider": { "@id": "https://agentu.co.uk/#org" } }
      }] },
    { "@type": "BlogPosting",
      "@id": "https://agentu.co.uk/insights/.../#article",
      "author":    { "@id": "https://agentu.co.uk/about/#person-daniel-byrne" },
      "publisher": { "@id": "https://agentu.co.uk/#org" } },
    { "@type": "Person",
      "@id": "https://agentu.co.uk/about/#person-daniel-byrne",
      "name": "Daniel Byrne",
      "worksFor": { "@id": "https://agentu.co.uk/#org" } }
  ]
}

Read what those @id references buy you. The article was written by that exact person, who works for that exact company, which sells that exact service at that exact price. An AI agent does not have to infer a single one of those links. They are declared.

Four things worth stealing from it.

One identifier per entity, forever. Answer engines deduplicate by identifier. Give the same human two @id values and you have created two people, each carrying half your authority. This is not hypothetical: we found exactly that defect in our own markup, and the graph confessed it within the hour.

Publish your prices. Every service we sell is an Offer with a real UnitPriceSpecification. Most agencies hide pricing behind "request a quote", and then wonder why they are absent from AI shortlists. An AI assistant cannot quote a price it cannot read. It will recommend the competitor whose price it could parse, and you will never see the enquiry that did not happen.

Only claim what you show. We emit a FAQPage node only on pages that visibly render those exact questions and answers. Schema describing content that isn't on the page is a parity violation, and the engines are getting good at catching it. Structured data is a claim about your page, not a second page.

Your taxonomy is an entity too. Our six Insights categories are not strings that happen to repeat. They are a DefinedTermSet of six addressable DefinedTerms, each with a definition an agent can fetch. Categories nobody outside your build can read are not a taxonomy, they are a variable name.

One more, because almost every site gets it wrong: a meta description and an entity description are two different jobs. One is a search snippet, written to earn a click. The other tells the graph what this thing fundamentally is. Same page, two readers, two strings. Every page on this site carries both, and they are deliberately not the same sentence. Collapse them into one and you have written something that half-serves a human and half-describes an entity, which is the most common way a perfectly good site ends up saying nothing precise about itself.

Layer 4: Serve the format agents actually prefer

HTML is a layout format that happens to carry text. Markdown is text. Given the choice, an AI agent would rather have the text.

So every key page on this site has a Markdown twin: the same URL with a .md suffix. /core.md, /agentu.md, and a .md for every article, every category and every author. Clean prose, no navigation, no markup to strip, a fraction of the tokens.

Three details decide whether that actually works, and two of them are headers:

• Advertise it. <link rel="alternate" type="text/markdown" href="…"> in the head, so an agent discovers the twin instead of guessing at it.
• Type it correctly. Content-Type: text/markdown, not text/plain and certainly not application/octet-stream.
• Open CORS. Access-Control-Allow-Origin: *. This is the one everybody misses. An agent running in a browser context that finds your rel=alternate link and then gets refused by CORS has been shown a door it cannot walk through.

And then go and read what your server actually returns, because headers are where this quietly breaks. Cloudflare, for one, joins the headers of every rule matching a URL rather than letting the most specific one win. Our blog rule silently matched the twins too, and every one of them shipped two contradictory cache policies for weeks. Nothing flagged it. curl -I your own twin. Trust what comes back, not what you think you configured.

Alongside the twins we publish an llms.txt manifest (linked from every page with rel="llms"), an /index.md index of every concept on the site, and a shared vocabulary at /context.jsonld that the twins' frontmatter resolves through, so the Markdown face and the HTML face are one graph described twice, not two descriptions that happen to agree.

That last part is a whole build of its own. We wrote it up here.

Layer 5: Be trusted, which mostly means be fast and be boring

Speed is not a UX nicety at this layer. It is a filter.

Humans have always applied it: as load time grows from one second to three, the probability a visitor leaves rises by 32%; from one to five seconds, by 90% (Google, 2017). An AI assistant assembling an answer in real time from a dozen sources applies the same test, faster, and without the guilt. A site that is slow to answer is treated as a site with no answer.

So: static files, edge delivery, content-hashed assets cached for a year, self-hosted fonts preloaded rather than fetched from someone else's CDN. Nothing clever. Just nothing in the way.

Security does the same job from the other side. A strict Content Security Policy with no unsafe-inline, HSTS preloaded, a security.txt, no database to breach, no third-party trackers, no cookie banner because there are no cookies to consent to. The most secure architecture is the one with almost nothing in it, and it happens to be the most legible one too. That is not a coincidence. Both audiences are rewarding the same thing: a site with nothing to hide behind.

Layer 6: Now stop it rotting

Everything above is true on the day you ship it. The interesting question is whether it is still true in March.

Rename an author, retire a page, change a price in one place and not the other, and your graph starts lying while the site still looks perfect to a human. Nobody gets an alert. You find out when the enquiries thin out.

So our build refuses to ship a lie. Three gates run on every deploy:

1. The graph must be whole. Every note carries a type and one absolute identifier, every link resolves, no two entities share an ID.
2. The sitemap must be true. If it disagrees with what was actually built, the build fails.
3. The prices, names and descriptions come from one file each. There is no second place to forget to update.

A failing build is not an inconvenience. It is the only thing standing between you and a site that quietly stops being readable.

The order is the whole point

If you take one thing from this: do these in order. JSON-LD on a site an agent cannot reach is decoration. An llms.txt on a site that needs JavaScript to render its own headline is a note pinned to a locked door.

Start here:

1. Turn JavaScript off and reload your homepage. What you can still read is what an AI agent sees. If that is a blank page, nothing else on this list matters yet.
2. Read your own structured data. Count the identifiers on your most important entity. More than one is a leak.
3. Find your prices in it. If they are not there, you are not on the shortlist.
4. Check your robots.txt is not quietly blocking the agents you want.
5. Then go build the graph.

The 40% of agentic AI projects Gartner expects to be cancelled by 2027 will not fail for lack of ambition (Gartner, 2025). They will fail on foundations nobody audited. The same is about to be true of a great many websites.

Our free agent-readiness diagnostic reads your site the way an AI agent does and tells you which of these layers you are failing, in about ten seconds. If you would rather we just fixed it, talk to us.

Being readable to an AI agent is not a feature you add. It is what is left when you take everything else away.