What llms.txt is
llms.txt is a proposal published by Jeremy Howard of Answer.AI in September 2024. It suggests that websites serve a Markdown file at /llms.txt that summarises the site and links to its most useful pages, so a language model or AI agent can understand the site without crawling and cleaning dozens of HTML pages.
Where robots.txt tells crawlers what they may not fetch, llms.txt is the opposite: a short, human-written guide to what is worth reading.
The format
The file starts with an H1 containing the site or project name, followed by a blockquote with a one-paragraph summary. Optional free text can follow. Then come H2 sections, each a Markdown list of links in the form - [Title](https://url): short note. A section named Optional marks links an agent can skip when it is short of context.
Some sites also publish llms-full.txt, a single file containing the full text of their documentation, and offer Markdown versions of pages by adding .md to the URL.
Does it help SEO or AI visibility?
llms.txt is not an official standard, and Google has said it does not use it for ranking or for AI Overviews. Some AI tools, coding assistants and documentation platforms do read it, and it costs little to maintain, so it is a reasonable low-effort addition rather than a ranking tactic.
What matters more for AI search today is that crawlers such as GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot are allowed to fetch your pages and that the content is readable without JavaScript.
Example
A minimal llms.txt:
# Example Co
> Example Co makes invoicing software for freelancers. Plans start at $9 a month.
## Docs
- [Getting started](https://example.com/docs/start): set up an account and send a first invoice
- [API reference](https://example.com/docs/api): REST endpoints and authentication
## Optional
- [Changelog](https://example.com/changelog)