llms.txt is a plain-text file meant to give language models a clean index of your website. Online it gets pitched in two ways: as the trick that gets you into ChatGPT, or as a straight copy of the spec. Neither helps you decide. The honest version: Google says in writing that its search engine ignores it, no major AI search engine has confirmed using it to pick sources, and it can still make sense for some sites because it costs almost nothing. Here is what it is, what really controls AI access to your site and when it is worth doing.
What is llms.txt?
A Markdown index of your site, written so a language model knows what to read first.
llms.txt is a proposal to publish, at yoursite.com/llms.txt, a Markdown file that sums up what the site is about and links to its key pages. Jeremy Howard put it forward on September 3, 2024, starting from a real problem: web pages are full of navigation, scripts and ads, and a model's context window is limited. A curated index saves an agent from crawling the whole site just to understand its structure.
The spec at llmstxt.org, whose current version was updated in August 2026, defines a fixed order. Only the title is required: an H1 with the name of the site or project. After that you can add a blockquote with a short summary, detail paragraphs without headings, H2 sections containing Markdown link lists, each link with an optional note, and a section called Optional for secondary links an agent can skip when it needs to save context.
The proposal adds two conventions: that each page offer a clean Markdown version at the same URL ending in .md, and that the file can also live in a subfolder, such as /docs/llms.txt, to cover only that section.
Do ChatGPT, Google or Perplexity actually use it?
Google says no; the others haven't confirmed it. Publishing one for their own docs doesn't mean their search reads yours.
Google is the only one that has answered in writing, and the answer is no. Its guide to optimizing for generative AI features, published in May 2026, lists llms.txt among the things you can ignore: you don't need AI text files, special markup or Markdown to appear in Google Search, including AI Overviews and AI Mode, because Google Search doesn't use them. On June 15, 2026 it added a clarification: maintaining these files for other services is fine, and it will neither help nor hurt your visibility or rankings on Google.
OpenAI, Anthropic and Perplexity haven't stated that their search products read llms.txt to decide what to cite. Their crawler documentation talks about robots.txt, not this file. It is true that OpenAI, Anthropic and Google publish llms.txt files for their own developer docs, as llmstxt.org points out, but that shows they offer it to AI coding tools, not that their search engines treat it as a signal.
Where it does show up is Chrome. Lighthouse includes an llms.txt audit in its agentic browsing category: it flags the page only if the server errors when fetching the file; a missing file returns a 404 and isn't flagged, because the file is optional. It checks that the file isn't broken, nothing more. It is not a ranking signal.
How is it different from robots.txt and sitemap.xml?
robots.txt grants permissions, sitemap.xml lists URLs to crawl and llms.txt summarizes the site for a model. Google only uses the first two.
All three sit at the domain root and are text files, but they do different jobs. robots.txt says which crawlers may enter and where. sitemap.xml lists the URLs you want search engines to discover. llms.txt summarizes the site so a model knows what to read. Only the first one controls access, and none replaces the others:
| File | What it does | Who reads it | Format | Effect on Google |
|---|---|---|---|---|
| robots.txt | Allows or blocks crawling per bot and per path | Googlebot, Bingbot, GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot and nearly every serious crawler | Plain text with User-agent, Allow and Disallow rules | Direct: block Googlebot and the page isn't crawled |
| sitemap.xml | Lists the URLs you want discovered and indexed | Search engines, Google and Bing above all | XML with URLs and last-modified dates | Helps discovery; doesn't guarantee indexing |
| llms.txt | Summarizes the site and points models and agents to key pages | Tools and agents that choose to read it; Lighthouse checks it doesn't error | Markdown: title, summary and link lists | None: Google says Search ignores it |
How do you create an llms.txt file? (template)
One Markdown text file at the root with a title, a summary and your important pages. Ten minutes of work.
Create a text file named llms.txt, upload it to your domain root so it answers at yoursite.com/llms.txt and check that the server returns a 200 status. This template follows the order in the spec; swap the names, URLs and notes for your own, line by line:
- # Your Company Name
- > Web development studio building corporate sites, online stores and custom software for small and mid-sized businesses.
- We work in English and Spanish. Each service page lists reference pricing.
- ## Services
- - [Web development](https://yoursite.com/services/web-development): corporate sites and landing pages, from strategy to launch.
- - [Online stores](https://yoursite.com/services/ecommerce): WooCommerce and Shopify builds with local payment methods.
- ## Work and guides
- - [Portfolio](https://yoursite.com/portfolio): published projects with scope and tech stack.
- - [Blog](https://yoursite.com/blog): practical guides on SEO, ecommerce and automation.
- ## Optional
- - [Privacy policy](https://yoursite.com/privacy): how personal data is handled.
What actually controls whether AI reads your site?
robots.txt per crawler, Search Console's generative AI setting and HTML that can be read without heavy JavaScript.
What decides whether AI reads your site is robots.txt, applied to each crawler separately. Every company runs several, with different purposes. Google uses Googlebot for Search, including its AI features, and the Google-Extended token to let you decide whether your content is used to train Gemini models and to ground answers in other systems; Google states that Google-Extended doesn't affect your inclusion or ranking in Search. OpenAI separates GPTBot, which collects content for model training, from OAI-SearchBot, which is what lets you appear in ChatGPT search answers. Perplexity uses PerplexityBot for its results, and Anthropic distinguishes ClaudeBot, for training, from Claude-SearchBot, for its search.
Watch for one exception: agents acting on a specific user request, such as ChatGPT-User or Perplexity-User, play by different rules. OpenAI says robots.txt rules may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores them, because a person initiated the visit.
Google also offers a dedicated control. Since August 31, 2026 every site has an option in Search Console Settings to exclude its links and content from Search generative AI features, without affecting its ranking in regular results. Underneath it all are the basics: important content in readable HTML that doesn't depend on heavy JavaScript to render text, an up-to-date sitemap.xml and structured data that matches what is visible.
Is it worth implementing?
Yes if you have documentation or a site that agents use; for everyone else it is a ten-minute detail, not a strategy.
It is worth it if your site has technical documentation, an API or content developers query through AI tools: there a clean index genuinely saves agents work, which is why OpenAI, Anthropic and Google publish one for their docs. It also makes sense if you already invest in technical hygiene and want Lighthouse's audit to come back clean.
For a small business with a corporate site or a store, it is an optional extra. It costs little to create, but it won't bring you Google traffic or guarantee ChatGPT citations. If you have one hour to spend on AI visibility, spend it first on robots.txt, on service pages that answer clearly and on Search Console's Generative AI report.
We publish one on beleafdesign.co, generated automatically from the site's content so it never goes stale, because Lighthouse checks it and because maintaining it costs us nothing. We don't sell it as a separate service, and you should be wary of anyone who does. For the bigger picture on earning visibility in AI search, see our AI SEO guide.
Frequently asked questions
Does llms.txt improve SEO?
Not on Google. Its official guide says Google Search ignores llms.txt and that having one neither helps nor hurts your visibility or rankings. No other major search engine has confirmed using it as a signal.
Where does the llms.txt file go?
At the domain root, so it answers at yoursite.com/llms.txt. The current version of the spec also allows placing it in a subfolder, such as /docs/llms.txt, to cover only that section's pages.
Can I block AI bots?
Yes, with robots.txt rules per crawler: GPTBot and ClaudeBot for training, OAI-SearchBot, PerplexityBot and Claude-SearchBot for search, and Google-Extended for Google's use in Gemini. Blocking the search crawlers removes you from those answers. Agents acting on a user's request may not honor robots.txt.
Does WordPress generate llms.txt automatically?
WordPress itself doesn't. Some SEO plugins offer it as an option, or you can upload the file manually via FTP or your host's file manager. Check that it returns a 200 status at the root.
What is llms-full.txt?
A file some documentation sites publish with all their content in Markdown in a single document, instead of just links. It isn't part of the spec published at llmstxt.org.
Summary: what to remember
- llms.txt is a Markdown index of your site for models and agents; it is not an official standard.
- Google says in writing that Search ignores it, and no other major AI search engine has confirmed using it to choose sources.
- Real AI access to your site is controlled with robots.txt per crawler and Search Console's generative AI setting.
Does your site let in the AI search engines you care about?
We review your robots.txt, indexing and content so Google, ChatGPT and Perplexity can read you, with no hype and no magic files.
See our SEO service
