What Is llms.txt? What It Does and How to Write One

Nobody is going to rank because of llms.txt. Worth saying before anything else, because there is now a small industry of tools and consultants selling the file as an AI visibility play, and the evidence does not back that up. Google’s John Mueller has publicly compared it to the old keywords meta tag, which is about as damning as search people get.

So what is llms.txt actually for, and why do we still ship one on every site we build? Because it costs an afternoon, it lives in your repo, and it does a few useful things that have nothing to do with rankings. Most of what gets sold as “AI readiness” is a monthly subscription to infrastructure you never touch, while this is a text file you write once and keep.

What Is llms.txt, Exactly

It’s a markdown file that sits at the root of your domain, at yoursite.com/llms.txt, and describes your site to a language model in plain language. Jeremy Howard at Answer.AI proposed the format in September 2024, and the structure is deliberately boring. The spec asks for an H1 with your name, a blockquote holding a one-sentence summary, then H2 sections containing lists of links, each with a short note about what’s behind it. No schema, no validator, no registry. If you can write a README, you can write an llms txt file.

The reasoning behind the format explains what the file is good for. Models work inside a context window, and HTML is an expensive way to spend it. Your homepage might be 400KB of markup and script tags wrapped around 600 words of real content, so a model that fetches the page burns most of its budget on things that mean nothing to it. The llms.txt file is the index card version: here is what this site is, here is where the substance lives, go read those.

What llms.txt Does Not Do

The honest starting point is the list of things people are being sold that the file does not deliver.

It has never been confirmed as a ranking or citation input. No major AI search product, OpenAI and Google and Perplexity included, has publicly said it reads llms.txt when deciding what to cite. Mueller’s comment on the subject went further than skepticism: “AFAIK none of the AI services have said they’re using LLMs.TXT (and you can tell when you look at your server logs that they don’t even check for it).” If an agency or a SaaS dashboard implies otherwise, ask where they got that, then watch what happens.

The server-log point held up when somebody finally measured it at scale. Ahrefs looked at 137,000 domains in May 2026 and found that 28% of them publish an llms.txt file, while 97% of those files were fetched by nothing at all that month. Among the small remainder that did see traffic, the largest share came from SEO audit tools and generic crawlers rather than the AI systems the file was written for. Adoption ran far ahead of anyone actually reading it.

People also confuse llms.txt with robots.txt, which does something else entirely. Google’s robots documentation describes a file that tells crawlers which URLs they may access, and it is honored by real infrastructure. Your llms.txt is voluntary, descriptive, and invisible to anything that does not go looking for it. You need both, plus an XML sitemap, which is what search engines actually use for discovery.

The last thing it won’t do is rescue a site that has nothing on it. Point the file at thin pages and you have built a very tidy index of nothing. If the underlying content is the problem, generative engine optimization is a much bigger conversation than a text file.

What llms.txt Is Actually Good For

Mostly, it’s the fastest brief you can hand anyone who has to work on your site. Paste yoursite.com/llms.txt into a chat, or point a coding agent at your repo, and whoever is on the other end gets an accurate map in about 300 words instead of reverse-engineering your navigation. We use ours internally far more than any outside tool ever has, and when a new contractor picks up a client project, the llms.txt is the first thing they read. It works the same way a single source of truth does for the rest of your operations, only smaller and cheaper.

Writing it is also a diagnostic. Try composing a one-sentence blockquote that says what your business does without using the word “solutions,” then try picking the twelve links that matter most out of your ninety-page site. Most owners can’t do it on the first pass, and the reason they can’t is usually the same reason the site doesn’t convert.

The third benefit is softer, so we’ll flag the uncertainty rather than oversell it. When a model does fetch the file, the summaries that come back are noticeably more accurate about positioning than the ones from a comparable site without it. Whether that ever turns into a citation, nobody can prove. But the failure mode without the file is a model reading your cookie banner and your footer and inventing the rest, and we have watched that land in front of clients who then wonder where the wrong description came from.

How to Write an llms.txt File Yourself

Here’s an llms txt example for a small business, written out rather than templated with brackets in it:

# Cascade Plumbing & Heating

> Family-owned plumbing and HVAC contractor serving Tacoma and
> Pierce County since 1998. Residential repair, water heater
> replacement, and emergency service, licensed in Washington
> (CASCAP*871QW).

## Services
- [Water heater replacement](https://example.com/water-heaters):
  Tank and tankless install, same-day in most of Pierce County.
- [Emergency plumbing](https://example.com/emergency): 24/7
  dispatch, typical arrival under 90 minutes.
- [Drain cleaning](https://example.com/drains): Camera
  inspection and hydro jetting for residential lines.

## About
- [Service area](https://example.com/service-area): Exact cities
  and ZIP codes covered, with after-hours boundaries.
- [Pricing](https://example.com/pricing): Flat-rate diagnostic
  fee and published ranges for common jobs.
- [Licensing and insurance](https://example.com/licensing):
  Washington contractor license, bond, and insurance details.

## Optional
- [Blog](https://example.com/blog): Maintenance guides and
  seasonal checklists.

What separates that from a link dump is entirely in the descriptions. “Same-day in most of Pierce County” carries information a model can do something with, while “fast, reliable service” carries none.

So load up the blockquote and the descriptions with the specifics nobody can infer from your page titles: license numbers, service radius, hours, and especially the work you don’t take. That last one matters more than owners expect, because a model with no stated boundaries will happily tell someone you handle commercial jobs you have never touched.

Use the Optional section the way it was designed, too. Anything under an H2 called Optional is explicitly skippable when context runs tight, so the blog archive and the press page belong there and your money pages don’t.

Generate It at Build Time

A hand-written llms.txt is accurate the day you write it and wrong four months later, so generate the thing during your build if your stack allows it. On lilagents.com ours comes out of src/pages/llms.txt.js, which reads our company profile from the repo and composes the file at build time, so editing the profile updates the output. That’s 41 lines of code standing between a living document and a stale one, and it’s the same instinct behind building a site you actually own rather than renting one.

If you’re on a hosted platform where you can’t add a route, a static file you update quarterly is a fine compromise and still beats paying someone monthly for the same thing.

llms-full.txt, and Whether You Need It

The companion convention is llms-full.txt, which is not an index but the entire body of your documentation or key pages concatenated into one markdown file, so a model can grab everything in a single fetch instead of following twenty links.

For a docs-heavy product that’s genuinely useful, which is why the llms full txt file shows up mostly on developer-facing sites where the documentation is the product. For a fifteen-page service business it’s largely redundant with just crawling the site, and you end up maintaining a second artifact for very little in return. Ship llms.txt first, then add the full version only if you have documentation worth concatenating.

Where llms.txt Costs You Something

We’d rather name the cost than pretend there isn’t one.

Doing it by hand takes longer than clicking a button. There are plenty of llms txt generator tools that will crawl your sitemap and produce a file in twenty seconds, and honestly the output is fine as a skeleton. What you get back is a list of your page titles, which is information the model already had. The boundaries, the specifics, and the judgment about which twelve links belong in there are the parts a crawler can’t produce, so the free generator saves you an hour and skips the part that mattered.

Build-time generation carries its own cost. Someone has to maintain that script, and if your platform won’t let you add a route, the option isn’t available to you at all. That’s the same trade-off that shows up everywhere in vendor lock-in: the cheap-feeling platform is cheap right up until you want to do something it didn’t anticipate.

Then there’s the possibility it never pays off in a way you can measure. There is no llms.txt column in GA4, and there isn’t going to be one. Our recommendation rests entirely on the math of an afternoon spent against a downside of zero, and anyone promising you attribution for this file is making it up. If you want work that actually moves AI visibility, that lives in content and structured data and crawlability, which is what our AI SEO work is built around.

Why We Keep Bringing This Up

The llms.txt file is the cheapest demonstration we know of for an argument we make constantly. The thing that makes your site legible to AI turns out to be a plain text file in your own repository, versioned and portable and free, while a whole category of vendors charges a monthly fee to manage your “AI presence” on a dashboard that goes dark the day you cancel. We ran into the same pattern when we scored the Fortune 500 on AI agent readiness and found that most of the basics were simply missing at companies with enormous budgets.

So write one. If you want a second pair of eyes on it, send it over and we’ll tell you what a model is likely to get wrong about your business. And if you sit down to write it and can’t work out which twelve links belong in the file, that’s the more useful conversation anyway.

---

Frequently Asked Questions

Is llms.txt worth adding to my site?

Yes, if you treat it as documentation rather than an SEO tactic. The file costs an afternoon, lives in your own repository, and gives anyone working on your site an accurate summary in a few hundred words. Just don’t expect it to change how you show up in ChatGPT or AI Overviews, because there’s no evidence it does that today.

Is an llms.txt generator good enough, or should I write it by hand?

Use a generator for the skeleton if it saves you time, then rewrite every description yourself. The generated version gives you structure and URLs, which is the easy half of the job. No crawler is going to write “we don’t do commercial jobs” for you, and that one line is worth more than the other twenty put together.

What is the difference between llms.txt and llms-full.txt?

The llms.txt file is an index, a short map of your site with links and a description for each one. The llms-full.txt file is the actual content of those pages concatenated into one large markdown document, which mostly makes sense for documentation-heavy products. Ship the index first and add the full version only if you have enough documentation to justify maintaining it.

Do I still need robots.txt and a sitemap?

Yes, and they matter considerably more than llms.txt does. Your robots.txt controls crawler access and is honored by real infrastructure, and your XML sitemap is what search engines use for discovery. The llms.txt file is additive and voluntary, so if you only have budget for one of the three, it isn’t this one.

Will llms.txt help me show up in ChatGPT or AI Overviews?

There’s no confirmed evidence it affects citation or ranking in any AI search product today, and Ahrefs found that 97% of published llms.txt files went unfetched in a single month of data. What helps is substantive content on a crawlable site with clean markup and accurate structured data. The file does a different job, improving the quality of what a model understands once it has already arrived.

Where does the llms.txt file go on my site?

At the root of your domain, so it resolves at yoursite.com/llms.txt with a plain text or markdown content type. If your stack supports custom routes, generate it during the build from a source of truth you already maintain so it can’t drift out of date. Otherwise a static file you review each quarter is perfectly reasonable.