In May 2026, Google published a guide on optimizing for AI search features. One line stood out: you do not need to create new machine-readable files, AI text files, markup, or Markdown to appear in generative AI search. The guide grouped llms.txt with content chunking and AI-specific rewriting as things site owners should not worry about.
Days earlier, the Chrome team shipped Lighthouse 13.3 with a new experimental category called Agentic Browsing. Among the audits: a check for the presence of an llms.txt file at the root of your domain. The documentation states that without this file, agents may spend more time crawling the site to understand its high-level structure and primary content.
Two teams inside the same company. One says you do not need it. The other builds a tool to verify you have it.
I have had llms.txt live on shinobis.com since January 2026. This is what I learned from being early, and what Google's contradiction reveals about where the agentic web is heading.
Two doors inside the same building
The Google Search team and the Chrome team are solving different problems. Search optimizes for ranking and retrieval. Chrome optimizes for how autonomous agents interact with websites in real time. These are not the same thing.
When the Search team says llms.txt is unnecessary, they mean it does not affect your position in AI Overviews or AI Mode. Their retrieval system uses the same index as traditional search. It does not read llms.txt. It does not need to.
When the Chrome team audits for llms.txt, they are evaluating whether a browser-based agent can efficiently understand your site without crawling every page. That is a completely different use case. An agent that lands on your site to perform a task (book a flight, compare prices, extract data) benefits from a file that summarizes what the site contains and where to find it.
John Mueller addressed the contradiction when asked about it directly. His response: it is not done for search. There is more to websites than just SEO. That single sentence is the key to understanding Google's position. llms.txt is not a search signal. It is an agent signal. And Google is building infrastructure for agents even while telling SEOs not to worry about it.
What the Lighthouse audit actually checks
The Agentic Browsing category in Lighthouse 13.3 does not produce a traditional score from 0 to 100. It reports a fractional pass ratio showing how many readiness checks your site passes. The category evaluates five areas: WebMCP integration, accessibility tree quality, cumulative layout shift, layout stability for agent interaction, and the presence of llms.txt.
For the llms.txt audit specifically, Lighthouse checks three things. First, whether the file exists at your domain root. A 404 response marks the audit as Not Applicable, not as a failure. Second, whether the file contains an H1 header. Third, whether the file is long enough to be useful and contains links to relevant content.
Server errors (500, 502, timeouts) flag the audit as failed. This matters because many CDN configurations and staging environments return errors for paths they do not recognize. A misconfigured server that blocks llms.txt is worse than not having one at all.
Google uses llms.txt on its own properties
In December 2025, an llms.txt file was discovered on Google's Search Central developer documentation. Multiple Google developer properties, including developer.chrome.com and web.dev, serve the file. When Mueller was asked about it, he responded with a vague acknowledgment and no further clarification.
This is not a contradiction. It is consistency. Google's developer documentation sites need to be readable by AI agents. The same logic applies to any site that publishes content for machines to process. The Search team is correct that it does not affect rankings. The Chrome team is correct that it helps agents. Both positions are true. They just apply to different systems.
What I implemented 5 months ago
I created my llms.txt file in January 2026. It contains a description of the site, a list of published posts with their URLs and one-line summaries, and links to the free tools (llms.txt Generator and GEO Tarot). The file is static, updated manually when new posts are published.
At the time, the reasoning was simple. Anthropic publishes an llms.txt on their domain. Cloudflare includes it in their Agent Readiness test. The specification at llmstxt.org had been gaining traction in the developer community. Whether Google cared about it was irrelevant. ChatGPT, Claude, and Perplexity have their own crawlers. A file that helps any of them understand my site faster is worth the 20 minutes it takes to create.
Five months later, my Cloudflare Agent Readiness score is 50 out of 100. Four of the ten standards the test evaluates apply to blogs. I implemented all four: llms.txt, content signals in robots.txt, Markdown negotiation for AI agents, and agent skills discovery. The other six (OAuth, MCP Server Card, WebMCP, API Catalog) are designed for SaaS platforms with public APIs.
Now Lighthouse audits for the same file I implemented when Google said it was unnecessary.
The pattern repeats
This is not the first time a standard moved from unnecessary to audited. Content Security Policy headers were optional for years before browsers started warning about their absence. HTTPS was a recommendation before Chrome began marking HTTP sites as not secure. Structured data was nice to have before Google built Rich Results around it.
The trajectory is always the same. An emerging convention appears. The official position is that it is not required. Developer tools start checking for it. Adoption reaches a tipping point. The official position updates.
llms.txt is at stage three right now. Lighthouse checks for it. The documentation calls it an emerging convention. The audit is experimental. But the infrastructure is in place.
What this means for your site
If you publish content and do not have an llms.txt file, now is the time. Not because it affects your Google rankings. It does not. But because the same company that told you it was unnecessary is now building tools to verify you have it. That tells you where the ecosystem is heading.
The file takes minutes to create. I built a free generator that creates one from any URL. No signup. No cost. Paste your domain and download the file.
The same company that said you did not need it now audits whether you have it. The question is not whether llms.txt matters. It is when you are going to implement it.