llms.txt is an optional Markdown file at your site's root that summarises your business and lists the pages AI tools should read. Google Search ignores it and few tools request it, so treat it as cheap housekeeping that comes after the basics that matter.
llms.txt is a short Markdown file at the root of a website, such as yoursite.com/llms.txt, that describes the business in plain words and links to the pages an AI tool should read first. Should your website have one? Only as low-cost housekeeping. Google Search says it ignores the file, few tools ever request it, and it will not lift your rankings or your visibility in AI Overviews. But it takes about an hour, cannot hurt you in Google, and a few tools, including a new audit in Google's Lighthouse, now check for it.
The idea came from Jeremy Howard of Answer.AI, who proposed it on 3 September 2024 with developers in mind: coding assistants pull library details from documentation sites, and one curated file is easier for a language model to use than a whole website. It is still a proposal published at llmstxt.org, not an adopted web standard.
- when Jeremy Howard of Answer.AI proposed llms.txt
- Sept 2024
- when Jeremy Howard of Answer.AI proposed llms.txt
- of sites served a valid llms.txt in the HTTP Archive's 2025 Web Almanac crawl
- 2.1%
- of sites served a valid llms.txt in the HTTP Archive's 2025 Web Almanac crawl
- of llms.txt files received no requests at all in May 2026, in an Ahrefs study of 137,210 domains
- 97%
- of llms.txt files received no requests at all in May 2026, in an Ahrefs study of 137,210 domains
- of the requests that did arrive came from bots Ahrefs classes as AI retrieval bots
- 1.1%
- of the requests that did arrive came from bots Ahrefs classes as AI retrieval bots
01
What goes in the file
The format is deliberately simple. The only required element is a top-level heading with the name of the site or business. The proposal then suggests a short summary in a blockquote, a few paragraphs of key details, and sections of Markdown links, each of which can carry a brief note on what the page covers. A section headed Optional holds links an AI tool can skip when it is short of space. Here is where it sits next to files you may already have.
| File | What it does | Who acts on it |
|---|---|---|
| robots.txt | Tells crawlers which parts of the site they may fetch | Google, OpenAI and Anthropic, to allow or block their bots |
| Structured data | Labels facts on a page, such as your business name, address or an article's author | Search engines; Google requires it to match the visible page |
| llms.txt | Offers a curated summary and reading list for language models | SEO audit tools more often than AI bots; Google Search ignores it |
02
What Google says: it ignores the file
Google's position has moved from an offhand comment to documentation. In April 2025 Google's John Mueller wrote on Reddit that, as far as he knew, none of the AI services had said they used the file, and that server logs showed their bots did not even check for it. He compared it to the old keywords meta tag: a site owner's own claim about what the site is about.
In May 2026 Google published a guide to its generative AI features in Search, and on 15 June added a note on llms.txt files. The guide says you do not need new machine-readable files, AI text files or Markdown to appear in Google Search, including its AI features.
Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them.
The same guide says it is fine to create and maintain the file for other services, and that optimising for generative AI search is still SEO. So if anyone sells you llms.txt as a route into AI Overviews or AI Mode, Google's own documentation says otherwise.
03
Who actually reads it
Almost nothing yet, on the best evidence available. In June 2026 Ahrefs, an SEO software company, published an analysis of server logs from 137,210 domains that use its web analytics. About 28% of them published an llms.txt file, which Ahrefs treats as an upper bound because its customers are more technical than the web at large, yet 97% of those files received no requests in May 2026. The largest share of the requests that did arrive came from SEO audit tools. AI bots made up 19.5%, and AI retrieval bots just 1.1%. GPTBot, OpenAI's training crawler, was the most active AI bot, and Claude Code, Anthropic's coding agent, fetched the files more often than any AI search or assistant bot.
The big AI companies have not said otherwise. The crawler pages published by OpenAI and Anthropic explain how to allow or block their bots through robots.txt and say nothing about reading a site's llms.txt, even though OpenAI publishes one for its own developer documentation.
The newest reader is Google's Chrome team. In May 2026, Lighthouse 13.3 added an experimental Agentic Browsing category that tests how well a site is built for AI agents to use, and one check looks for llms.txt. Chrome's documentation says that without it, agents may spend longer crawling a site to learn its structure. That concerns agents navigating your site, not search rankings.
04
So should a small business publish one?
Across the wider web, adoption is thin. The HTTP Archive's 2025 Web Almanac found valid llms.txt files on about 2.1% of the sites it crawled, and 39.6% of those carried comments added by the All in One SEO plugin for WordPress, so many owners may never have chosen to publish one. Yoast SEO added a generator in June 2025. A WordPress site may already have a file nobody has read.
Our view: treat llms.txt as cheap housekeeping, not a growth tactic. It cannot hurt you in Google, may help the tools that look for it, and forces a useful decision about which pages define your business. But it comes after the basics that demonstrably matter: pages Google can crawl and index, accurate structured data and a complete Google Business Profile, all covered in our guide to Answer Engine Optimization.
- Do it now if your SEO plugin already offers it and you only need to review the output, or if you run a clinic, school or hotel whose facts an agent might one day use to compare or book.
- It can wait if Search Console shows important pages that are not indexed, your key pages are slow, or your Business Profile still lists old hours.
- Never use it to block AI crawlers, which is the job of robots.txt, or fill it with claims and keywords that do not appear on your pages.
05
How to write a good one
Write it by hand, or edit what your plugin generates. A file that lists every recent post misses the point: the value is in curation, so choose the ten to twenty pages that best explain what you do. Here is the shape we suggest, for a hypothetical dental clinic in Kochi.
- Heading: the clinic's trading name, exactly as it appears on the website, signboard and Google Business Profile.
- Summary: one sentence on what it does and where, for example family and cosmetic dentistry in Kakkanad, open six days a week.
- Key facts: address, phone and WhatsApp numbers, opening hours, languages spoken and how to book, each matching the website word for word.
- Services: one link per treatment page, with a short note on who it suits and the price if the site publishes one.
- Policies: links to the pages on payment options, insurance, cancellations and privacy.
- Optional: the blog, careers page and older news, which an AI tool can skip.
Save it as plain text named llms.txt in your site's root folder, check that the address loads, and review it whenever prices, hours or services change. An out-of-date file repeats old facts to any tool that trusts it.
What to do this week
- Visit yoursite.com/llms.txt to see whether a file already exists, and read it if it does.
- Replace a generic plugin file with a curated one, or switch it off if nobody will maintain it.
- Check every fact in it against your website, structured data and Google Business Profile.
- Confirm the address returns the file or a clean 404, never a server error.
- Keep robots.txt as your real control over which AI crawlers may visit.
- Put the bigger effort into crawlable, useful pages, which are what Google's AI features rely on.
Want a second opinion? Our free audit covers your llms.txt along with the basics that matter more.