What is llms.txt?
llms.txt is a plain-markdown file served at a website’s root (example.com/llms.txt) that gives AI systems a curated, LLM-friendly map of the site: what it is, and which pages best represent it. Where robots.txt tells crawlers what they may not access, llms.txt tells AI crawlers what is most worth reading.
The format, proposed at llmstxt.org, is deliberately simple: an H1 with the site name, a one-paragraph summary blockquote, then H2 sections of annotated links. A companion llms-full.txt can inline the full content of the linked pages for engines that prefer one fetch. Because it is markdown, the file doubles as a human-readable site summary.
llms.txt earns its place through economics: LLM crawlers spend a limited budget per site and parse imperfectly. A curated map spends that budget on your best pages, described in your own words — rather than leaving an engine to infer your site’s shape from whatever navigation it happened to parse.
It complements rather than replaces the classic crawl files: robots.txt still governs access, sitemap.xml still enumerates URLs exhaustively, and llms.txt curates. This site ships all three — the llms.txt at /llms.txt is generated by the toolkit’s own generator.
Common questions
Does llms.txt actually help AI visibility?
It is an inexpensive positive signal, not a silver bullet. Adoption by AI crawlers is still uneven, but the file costs minutes to generate, cannot hurt, is read by a growing set of agents and tools, and — like structured data in the early days of rich results — rewards the sites that shipped it before it became table stakes.
Where does llms.txt go?
At the web root, next to robots.txt: https://example.com/llms.txt. If you publish the optional llms-full.txt, it sits at the root as well. Serve both as plain text or markdown with a 200 status.