llms.txt: what it is, what it is not, and whether you need one

Two claims are made about llms.txt, and both are wrong.

The first is that it is a ranking factor for AI search — that publishing one gets you cited. Nobody has shown this, and no engine has committed to reading the file. The second is that it is therefore pointless. That one is wrong for a subtler reason, and it is the interesting half.

What the file actually is

llms.txt is a convention proposed in 2024 by Jeremy Howard: a plain-text file at the root of a site — /llms.txt — that says what the site is and links to the pages carrying the substance. It is Markdown, and the shape is fixed:

# Acme

> Acme builds accounting software for UK construction firms.

Optional paragraphs of context that do not fit the sentence above.

## Product
- [Pricing](https://acme.com/pricing): Plans, limits and what each tier includes.
- [Integrations](https://acme.com/integrations): The systems Acme reads from.

## Documentation
- [API reference](https://acme.com/docs/api): Endpoints, auth and rate limits.

## Optional
- [Changelog](https://acme.com/changelog): Release notes back to 2021.

An H1 with the name. A blockquote with one sentence defining the thing. Then ## sections of links, each with a short note saying what is in it. A section called Optional means “skip this if you are short of room”.

That is the whole specification. It takes twenty minutes, most of which is spent deciding what to leave out.

What it is not

Not a sitemap. A sitemap is an exhaustive machine-readable list of every URL, written for a crawler deciding what to fetch. llms.txt is a short, opinionated, human-written summary for a reader deciding what matters. A sitemap with twelve entries would be broken. An llms.txt with four hundred is useless. They are not versions of each other.

Not robots.txt. It grants nothing and forbids nothing. If your robots.txt shuts an engine out, publishing this file changes exactly nothing — the engine that cannot read your pages cannot read this one either. Check that first; it is the more important finding by a wide margin.

Not a commitment by anyone. No major engine has published a promise to fetch /llms.txt and treat it as authoritative. Adoption is real among developer-documentation tools and real among people who write about AI search; it is not the same thing as a consumer.

The case that actually stands up

Here is the thing worth understanding about how a model describes your company. Everything it knows, it inferred. It read your navigation, your hero copy, a pricing table and a footer, and worked out what you sell from the arrangement of those things. That inference is usually right.

When it is wrong, it is wrong in a specific and recognisable way: a product you discontinued two years ago is still in the summary; a positioning line from a page you rewrote survives; a company with a similar name gets folded into yours. These are not hallucinations in the dramatic sense. They are reasonable readings of a site that never stated the thing plainly anywhere.

llms.txt is the flattest available way to state it plainly. No navigation, no styling, no marketing structure, no inference required. And the argument for publishing it does not depend on any engine adopting the convention — it is a page on your site, it is crawled like any other page, and it says the thing you most want understood in the least ambiguous form available.

The downside is a text file nobody reads. Weigh that against twenty minutes.

The one line that matters

Almost all of the value is in the blockquote. It is the sentence most likely to be quoted back as the definition of what you are, so write it as a definition rather than as a headline.

• Lead with the entity name and a category: Acme is accounting software for UK construction firms.
• Name who it is for. “Software for construction” and “software for construction finance teams” get retrieved for different questions.
• Cut the adjectives. Leading, innovative and next-generation carry no information and displace words that do.
• Use the same sentence everywhere — your About page, your schema markup, this file. Four paraphrases across four pages is exactly the ambiguity you are trying to remove.

Where people get it wrong

• Listing everything. The file is a set of priorities. Dumping the sitemap into it discards the only thing it contributes.
• Writing marketing copy in the blockquote. If the sentence would work on a billboard, it is not a definition.
• Publishing and forgetting. A file that describes last year’s product is worse than none, because it is stated flatly enough to be believed.
• Serving it as HTML, or behind a redirect. It is text/plain at the root, fetched directly.
• Treating it as the work. It is a label on the box. Whether the contents are worth citing is a separate question and a much larger one.

Generate a draft, then edit it

The tedious nine-tenths of this job is reading your own site and grouping the URLs. The llms.txt generator does that part: it crawls up to fourteen pages, takes your name from your schema markup where you have it, files pages into sections by path, and writes the file in the conventional shape. Every description is your own meta description or heading — nothing is summarised by a model, because a plausible sentence about a page half-read is precisely what a file claiming to be authoritative must not contain.

What it hands you is a draft. Which pages matter is a judgement about your business, and a crawler does not have one.