An llms.txt file is a short markdown file, published at yourdomain.com/llms.txt, that summarizes a website and links to its most important pages for AI systems to read. It follows a fixed format proposed in September 2024. The honest part marketing teams need to hear first: almost no AI system currently reads it.
Key takeaways
- llms.txt is a plain markdown file at domain.com/llms.txt that lists a site’s key pages for AI systems, similar in spirit to a sitemap but written for language models instead of search crawlers.
- Adoption is real but small: 8.7% of the top 1,000 websites now publish an llms.txt file.[3]
- Usage is close to zero. Google’s John Mueller has confirmed that no AI system currently uses llms.txt, and only 0.1% of AI bot requests actually hit the file.[5]
- 97% of published llms.txt files received zero AI requests in May 2026, even as the number of files grew 8.8 times year over year.[4]
- The effort to create one is small, roughly 30 to 60 minutes, which makes it reasonable as cheap insurance even while it delivers nothing measurable today.
What is llms.txt?
llms.txt is a plain text file written in markdown and placed at the root of a domain, at exactly yourdomain.com/llms.txt. It gives an AI system a short, curated overview of what a site is and where its most useful content lives, written for a machine to parse quickly. Jeremy Howard of Answer.AI proposed it as a standard in September 2024,[2] and the specification is maintained at llmstxt.org.[1]
The everyday analogy is a lobby directory board. It does not run the building, but it tells a visitor who does not know the layout which floor to visit, saving them from checking every door. llms.txt plays the same role for an AI crawler landing on your domain with a limited context window: here is who we are, here is where to look first. What it does not do is control access, block crawlers or guarantee indexing. It is a suggestion, not an instruction, and that distinction matters more here than for older web standards.
llms.txt versus robots.txt and sitemaps
Marketing teams often assume llms.txt sits in the same family as robots.txt and sitemap.xml, since all three are small text files at the domain root. The purpose and authority behind each one differ.
| Dimension | robots.txt | sitemap.xml | llms.txt |
|---|---|---|---|
| Purpose | Tell crawlers what they may or may not access | List indexable pages so search engines can find and crawl them | Summarize the site and point AI systems to key pages to read |
| Who it addresses | Search engine crawlers and bots generally | Search engine crawlers | AI assistants and LLM based crawlers, in theory |
| Format | Plain text, fixed syntax (User-agent, Disallow, Allow) | XML, defined schema | Markdown, loose structure defined by the llms.txt spec |
| Effect | Enforced by compliant crawlers, shapes what gets crawled | An indexing signal, not a guarantee of inclusion | No enforcement mechanism, purely informational |
| Whether it is binding | Yes, a directive crawlers are expected to respect | No, a hint search engines may or may not follow | No, and currently not confirmed to be read by major AI systems |
The practical takeaway for a marketing lead briefing IT: robots.txt and sitemap.xml sit on infrastructure search engines have used for two decades. llms.txt sits on a proposal barely two years old, with no confirmed adoption by the systems it targets, so it should be briefed and budgeted very differently.
How an llms.txt file is structured
The specification is short enough for any content or IT person to implement correctly in one sitting. The file lives at exactly domain.com/llms.txt, at the root, not in a subfolder. It is plain markdown, no HTML, no JavaScript, and it follows a fixed order.
An H1 with the project or company name is the only required element. Directly under it comes a blockquote with a one or two sentence summary. Optional paragraphs without headings can follow. Then come H2 sections, each a list of links in the format [link text](URL): optional note, grouping related pages such as docs, blog or pricing. An optional “Optional” section holds secondary material AI systems can skip if short on context space. There is also an llms-full.txt variant, which inlines the full text of linked pages instead of just links, useful for smaller sites.
Size guidance matters because AI systems have limited context windows: keep the file under 50 KB, ideally under 15 KB, so it can be read in full rather than truncated.
A minimal, correctly structured example:
# neuroflash
> neuroflash is an AI marketing platform for content creation and AI visibility research.
neuroflash helps marketing teams plan, write and measure content for both traditional search engines and AI answer engines.
## Docs
- [Getting Started](https://neuroflash.com/docs/getting-started): setup and first steps
- [API Reference](https://neuroflash.com/docs/api): endpoint documentation
## Optional
- [Careers](https://neuroflash.com/careers): open rolesWhat belongs in it, and what does not
Include the company or product name, a one or two sentence summary an outsider would understand immediately, and links to the pages you would want an AI system to read first: core documentation, your main product or pricing page, and one or two evergreen guides.
Leave out a full dump of every sitemap URL, since that defeats the purpose of a curated file. Leave out marketing adjectives, temporary campaign pages, anything behind a login wall, and duplicate content that lives elsewhere on the site. Treat it like a one page backgrounder handed to a new analyst: dense with facts, no dead links.
What llms.txt actually delivers today
This is the part most guides gloss over, and it deserves the same directness as the rest of the file.
Google has addressed this directly. John Mueller has confirmed that no AI system currently uses llms.txt.[5] That is Google’s own statement, not a marketing team’s guess. The usage data backs that up. Of 62,100 AI bot requests analyzed, only 84 went to an llms.txt file, 0.1%.[5] Publishing grew fast even as usage did not: files grew from 4,088 in June 2025 to 36,120 in May 2026, an 8.8 times increase, yet 97% received zero AI requests that month.[4] Among the top 1,000 websites, 8.7% now publish one.[3]
Two more data points explain why the file struggles even where it is read. AI engines do not agree on what to cite: top 100 citation sources overlap only 16% to 59% between engines,[6] so a file listing your preferred pages cannot fix a landscape where each engine draws from a different source pool. What correlates with staying cited is freshness: pages not updated quarterly are three times more likely to lose their citation,[7] a signal about maintenance, not a manifest file.
None of this makes llms.txt a wasted afternoon. The honest expectation is zero measurable effect today, with the caveat that today is not forever.
So is it worth doing?
Yes, and the reasoning is simple. Creating a correct llms.txt file takes 30 to 60 minutes, a trivial cost against any scenario where adoption improves. The standard is young, proposed less than two years ago,[2] and writing a clear business summary and organizing your best pages into a short list is useful on its own, regardless of whether any AI system reads the file.
Treat it as cheap insurance: publish it, keep it accurate, revisit it a few times a year, and do not treat it as a strategy. Do not shift budget away from what demonstrably works today, such as structuring content so answer engines can parse and cite it, keeping pages fresh, and understanding the real questions your audience asks AI systems. That matters because AI referral traffic, while small in volume, is disproportionately valuable: ChatGPT traffic accounts for roughly 0.5% of visits on some sites yet drives about 12.1% of signups.[8] Our LLM SEO guide covers what actually works, the AI Visibility 2026 guide has the full playbook, and answer engine optimization is the broader discipline llms.txt sits inside.
Why an llms.txt file cannot tell you what your audience actually asks
An llms.txt file lists your content. It says nothing about whether that content actually answers the questions your audience asks an AI system before they reach your site. Keyword tools do not solve this either, because people do not type keywords into ChatGPT or Perplexity. They type whole situations, with context, constraints and a follow up question already forming in their head.
neuroflash’s Digital Twins are built for exactly that gap: direct access to more than 1,000,000 real human profiles you can query yourself. How would this audience actually research a decision like the one your product solves? Which prompt comes first? Which follow up question comes next? Which criteria decide between two options? Twins reach 85 to 98% predictive accuracy compared to roughly 55% for generic AI tools, a gap validated across more than 80 academic studies, with results back in minutes instead of the 4 to 8 weeks a classic survey takes.
Once you know the real prompts and decision criteria, automated workflows take over: writing new articles that are AI-optimized from the start, and reworking existing articles so answer engines can parse and cite them.
- Query a panel of more than 1,000,000 real human profiles instead of guessing at prompts
- See the follow up questions and decision criteria that come after the first one
- Get answers in minutes instead of the weeks a traditional survey takes
- Feed what you learn into workflows that write and rework articles for answer engines
FAQ
What is llms.txt used for?
It gives AI systems a short, structured summary of a website and links to its key pages, so a model does not have to guess at site structure. Adoption by AI systems remains very low as of mid 2026.
Does ChatGPT read llms.txt?
There is no confirmed evidence that ChatGPT, or any other major AI system, actively uses llms.txt files. Google’s John Mueller has stated that no AI system currently uses it, and independent analysis found only 0.1% of AI bot requests reached the file.
How do I create an llms.txt file?
Write a markdown file with an H1 for your company name, a one or two sentence blockquote summary, and H2 sections listing key pages as links with a short note each. Save it as llms.txt and publish it at the root of your domain.
Where do I put the llms.txt file?
At the exact root of your domain, as yourdomain.com/llms.txt, the same location convention used by robots.txt and sitemap.xml.
Is llms.txt required for AI SEO?
No. It is not required, and current data shows close to no measurable effect on citations today. It is a low cost addition worth doing, but budget for AI visibility should go toward content structure, freshness and real audience questions, which have demonstrated impact.
My Take
llms.txt is worth 30 to 60 minutes of someone’s time and nothing more than that for now. Publish one because it costs almost nothing and the standard may still gain ground. Do not mistake publishing it for an AI visibility strategy, and do not let it absorb budget that should go toward content structure and freshness, which the data shows actually correlates with getting cited.
References
[1] Answer.AI, llmstxt.org: “Official specification of the llms.txt standard, proposed by Jeremy Howard.” https://llmstxt.org/
[2] Search Engine Land (2025): “Jeremy Howard of Answer.AI proposed llms.txt as a standard in September 2024.” https://searchengineland.com/llms-txt-proposed-standard-453676
[3] Rankability (2026): “8.7% of the top 1,000 websites publish an llms.txt file.” https://www.rankability.com/data/llms-txt-adoption/
[4] ppc.land (2026): “The number of llms.txt files grew from 4,088 in June 2025 to 36,120 in May 2026, yet 97% of those files received zero AI requests in May 2026.” https://ppc.land/llms-txt-adoption-rises-8-8x-but-97-of-files-get-zero-ai-requests/
[5] Kai Spriestersbach (2026): “Only 84 of 62,100 AI bot requests went to llms.txt, that is 0.1%. Google confirmed through John Mueller that no AI system currently uses llms.txt.” https://medium.com/@kaispriestersbach/the-llms-txt-is-dead-more-precisely-a-dud-ab7bee4f469c
[6] BrightEdge (2026): “The overlap of top 100 citation sources between AI engines is only 16 to 59%.” https://www.brightedge.com/resources/weekly-ai-search-insights/ai-search-same-brands-different-sources
[7] AirOps (2026): “Pages not updated quarterly are three times more likely to lose citations.” https://www.airops.com/report/the-2026-state-of-ai-search
[8] Ahrefs (2025): “ChatGPT traffic accounts for about 0.5% of visits but generates about 12.1% of signups.” https://ahrefs.com/blog/chat-gpt-traffic
