Every few months a new file appears that you are supposed to add to your site or fall behind. llms.txt is the current one. It is worth understanding, and it is worth being honest about what it does not do.

Key takeaways

• llms.txt is a community proposal, not a ratified standard, and no major AI provider has publicly confirmed they use it.
• It is a markdown file that points at your best content in a clean, parseable form. It does not control access.
• It is not a substitute for robots.txt — the two solve opposite problems.
• Cost to add: roughly an hour. Expected return today: uncertain. That combination makes it a reasonable low-priority bet, not an emergency.

What is llms.txt?

llms.txt is a proposed convention for a markdown file at your domain root that gives language models a curated, clean-text map of your site. The idea is that instead of parsing your navigation and guessing what matters, a model reads one file listing your key pages with short descriptions.

The format is deliberately simple: an H1 with your site name, a blockquote summary, then sections of markdown links with one-line descriptions.

What does it actually look like?

\\\`

FixRoast

> AI landing page analysis with real Core Web Vitals data, plus AEO and GEO

> scoring for visibility in ChatGPT, Perplexity and Google AI Overviews.

Core pages

• Landing page roast: Eight-dimension

scoring with specific fixes for design, copy, UX and performance.

• AI search visibility: AEO and GEO scoring,

plus tracking of whether AI engines name your brand.

Free tools

• AI citation check: Grades whether

a page is structurally citable. No signup.

Optional

• Pricing: Plans and credit costs.

\\\`

That is essentially the whole specification. Sections under ## Optional are marked as skippable when a model has a limited context budget.

How is it different from robots.txt?

They are frequently confused, and the confusion matters because only one of them actually does anything today.

If your actual goal is to stop AI crawlers using your content, llms.txt does nothing for you. You need robots.txt directives targeting the specific crawler user-agents, and you should know that compliance is voluntary and varies between providers.

Should you add one?

Add one if the cost is genuinely near zero for you and you are already producing structured content. Skip it if you are choosing between this and fixing something that demonstrably affects visibility today. The expected value is real but unproven, and there are higher-return uses of the same hour.

Here is the honest ordering. If you have not done the things above llms.txt in this list, do those first:

1. Make your pages extractable — question-shaped headings with direct answers underneath. This affects every engine, today.
2. Add Organization and Article schema — widely consumed, unambiguous, well documented.
3. Add FAQPage schema to your question-shaped content.
4. Fix Core Web Vitals — slow pages get crawled less and convert worse regardless.
5. Then add llms.txt.

How do you create one?

1. List the 10–20 pages you would want an assistant to know about.
2. Write a one-sentence description of each. Describe what the page answers, not what it is called.
3. Group them into sections with ## headings.
4. Put genuinely secondary material under ## Optional.
5. Save as markdown at https://yourdomain.com/llms.txt.
6. Verify it loads as text/plain or text/markdown, not HTML.

Some sites also publish llms-full.txt containing the full text of key pages. That is a heavier commitment and duplicates content you already serve, so treat it as optional.

What this does not solve

llms.txt will not make a thin page citable. If the underlying content does not contain a clear answer to a real question, listing it in a manifest changes nothing — you have simply pointed an engine at a page it will then decline to quote.

The order of operations is: write something worth citing, structure it so it can be extracted, identify yourself so it can be attributed, and only then worry about discovery manifests.

You can check whether your pages clear the first three bars with our free AI citation check — it reads your live HTML and grades schema, answer structure, trust signals and brand identity in a few seconds.

Related reading: How to get cited by ChatGPT.