Agent-ready website checklist: what actually works (2026)
A website is agent-ready when AI search engines and AI agents can fetch your pages, understand them and complete a task on them. Google says you need no special AI files for this: it starts with an indexed page, text that sits in the HTML, a robots.txt that lets in the relevant crawlers for each AI service, and buttons and forms a machine can operate.
A prospect asks ChatGPT which accounting firm in their area also handles payroll, and gets three names back. Your firm's chance of being one of them depends partly on choices you make on your own website. Google itself calls a few popular checklist items unnecessary.
Below are ten points, ordered by what an AI does with your site: fetch it, understand it and act on it.
What is an agent-ready website?
An agent-ready website is a site that AI systems can read and use unaided. That covers AI search engines such as ChatGPT search and Google AI Mode, and AI agents that fill in a form or book an appointment for someone.
"Agent-ready" is an umbrella term; there is no official standard behind it. For how it relates to SEO, see SEO vs GEO vs Agent-Ready.
Can AI search engines read your website?
An AI search engine can only read your website if its crawler may fetch your pages, so check the rules for each AI service and put your key information in the HTML your server sends. Three points:
1. Allow the search crawlers in your robots.txt. AI companies run separate crawlers for search and for model training, and you can allow or block each one on its own. At OpenAI, OAI-SearchBot finds pages for ChatGPT, and OpenAI's documentation states that sites opted out of it "will not be shown in ChatGPT search answers". GPTBot collects text for training. Anthropic splits it the same way: Claude-SearchBot for search results, Claude-User when someone asks Claude a question, ClaudeBot for training. Google-Extended only governs Google's use of your content for Gemini and, according to Google, has no effect on inclusion in Google Search.
Check it yourself: open yourdomain.com/robots.txt in your browser. A crawler follows only the group of rules that most specifically matches its name and ignores the rest, Google explains. Disallow: / under OAI-SearchBot or Claude-SearchBot shuts that search engine out. Under User-agent: * it applies to every crawler without a group of its own.
2. Put your key text in the HTML. In December 2024 Vercel found that "none of the major AI crawlers currently render JavaScript". Gemini runs on Googlebot's infrastructure and does execute JavaScript. Text that only appears in the browser is invisible to those crawlers.
Check it yourself: on an important page, press Ctrl+U (view source; Cmd+Option+U on a Mac) and use Ctrl+F to search for your phone number or the name of your main service. The source shows your server's first response, including data sent along as JSON; Vercel notes that such content in the initial response may still be indexed. If it is missing there, a crawler that skips JavaScript may miss it.
3. Make sure Google may show your page. For AI Overviews and AI Mode, Google requires that a page is "indexed and eligible to be shown in Google Search with a snippet", and there are "no additional technical requirements". Search Console has a setting for generative AI features that defaults to inclusion; it sits under Settings, as Google describes.
These three points make a mention possible without making it certain. Google's guide to generative AI in Search says it plainly: "Indexing and serving aren't guaranteed." The reverse does hold: if you block OAI-SearchBot, OpenAI says your site will not appear in ChatGPT search answers.
Can an AI understand what your business does?
You make it easier for an AI to interpret your business correctly when the facts that matter are written as plain text on a fixed page and your structured data matches them. Two points:
4. One page per service, with the facts in text. What you do, for whom, in which area, how a project runs, roughly what it costs and how to get in touch. Google says that creating content people find "unique, compelling, and useful" will likely influence your presence in generative AI search in the long run more than any of the other suggestions in its guide. It advises creating that content yourself, "based on what you know about the topic".
5. Structured data that matches what the page says. Structured data is a block of code that tells a machine, in fixed fields, who you are and what you offer. Without it, says schema.org, search engines have "a limited understanding" of what a page is about. Google recommends the JSON-LD format and sets one firm rule: "don't add structured data about information that is not visible to the user", even when that information is accurate (Google Search Central). Start with Organization or LocalBusiness on your homepage and Service on each service page. Google says this is not required for generative AI search, and still calls it "a good idea" for regular SEO.
Check it yourself: paste your URL into Google's Rich Results Test. It shows what Google finds on the page and flags any errors.
Do you need an llms.txt file?
You do not need an llms.txt file for Google Search: on 15 June 2026 Google wrote in the same guide that Search ignores the file and that it will "neither harm nor help" your visibility or rankings. Maintaining one for other services that use it is, in Google's words, "completely fine".
6. llms.txt is an optional proposal. The file is a proposal by Jeremy Howard from September 2024: a markdown file at /llms.txt that helps agents use a website. Only a heading with your site's name is required. Chrome's Lighthouse tool has an audit for it, calls it "an emerging convention" and says the file is "optional at the moment". Create one if you have a concrete use for it, and expect no effect in Google. How to build it is in our llms.txt guide.
How AI-ready is your business?
Take the free AI Readiness Quiz: 7 questions, 2 minutes.
Take the quiz →Can an AI agent complete a task on your website?
An AI agent completes a task on your website more easily when buttons, links and form fields make clear in the code what they are. Modern browser agents can combine the code and the accessibility tree (the layer screen readers use) with a screenshot, write Kasper Kulikowski and Omkar More in Build agent-friendly websites on web.dev, Google's developer platform. Four points:
7. Real buttons and links. Use <button> and <a>, and avoid building a button out of a styled <div>.
8. A label on every form field. web.dev recommends giving each field a <label> tied to it with the for attribute. That way an agent knows the "Phone number" field wants a phone number.
9. A stable page without transparent layers. web.dev warns against "transparent overlays that hide interactive elements" and against buttons that move position from page to page.
Check it yourself: work through your contact form using only the Tab key and Enter. If you get stuck, there is probably something in the code that will trip up an agent as well. Treat this as a rule of thumb; a full test takes more.
10. A dedicated entrance for machines, as the final step. WebMCP lets you declare which tasks an agent may perform on your site, and web.dev advises asking for confirmation before sensitive actions such as a purchase. Google's guide describes new agent protocols as "emerging". What is possible today is in WebMCP: what it is and how it works.
What can you skip?
You can skip anything that promises a special file, special markup or a special writing style will get you into AI answers. Google's guide says: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search." Three common checklist items:
- FAQ schema for expandable questions in Google. According to the Google Search Central changelog, Google has stopped showing that result since 7 May 2026. Questions and answers on your page still help your readers.
- Chopping text into small chunks for AI. Google: "There's no requirement to break your content into tiny pieces for AI to better understand it."
- Collecting inauthentic mentions on other sites. Google: "Seeking inauthentic 'mentions' across the web isn't as helpful as it might seem."
What does this look like on mindsora.io?
What we measured from the outside on our own site (own measurement, 17 October 2026):
- robots.txt allows OAI-SearchBot, GPTBot, ClaudeBot and Google-Extended by name. For crawlers without a group of their own it blocks
/api/, with one exception:/api/services. Corrected on 14 November 2026: this said the file blocks/api/, without adding that the rule does not apply to the crawlers that have a group of their own. See the check under step 1. - Our llms.txt file is live, as an optional feature following the llmstxt.org proposal.
- The service catalogue at /api/services returns our services as JSON, with prices in euros excluding VAT and the date of the last price change.
- The homepage carries structured data of the types Organization, Service, OfferCatalog, FAQPage and HowTo.
- Our Agent-Ready Grader gave mindsora.io 98 out of 100. Semantic HTML scored 8 out of 10, and the grader recommends more
<header>,<main>,<nav>and<article>there.
That grader is our own tool. It gives llms.txt 20 of the 100 points. That is our own weighting; Google Search does not use the file.
The Agent-Ready Audit (€697 one-off) is listed in our services overview.
Where should you start?
Start with points 1 to 3, because if a crawler cannot fetch your page, nothing else matters. Then points 4 and 5, which also help people and Google. Points 7 to 9 go to your web developer as a brief, and llms.txt and a machine entrance come last.
To find out which points will pay off most for your business, take the free AI Readiness Scan.
Ready to automate?
Book a free consultation and discover what AI automation can do for your business. Or take the 2-minute AI Readiness Quiz first.
Book a call