A quiet convention called llms.txt is gaining traction across the web — and it could change how AI systems consume your product's documentation and content.

llms.txt: The Emerging Standard for AI-Readable Websites

There is a file sitting at the root of a growing number of websites — llms.txt — and if you build software products, you should know what it is and why it exists.

The Problem It Solves

Large language models are increasingly used as the first point of contact between a developer and a piece of software. Instead of reading documentation linearly, engineers paste a library's docs into ChatGPT, query a Cursor AI sidebar, or let an agent autonomously browse API references. The issue is that most websites are built for human eyes: navigation menus, cookie banners, marketing copy, and JavaScript-rendered content that an LLM either cannot parse or has to wade through to find anything useful.

The signal-to-noise ratio is terrible. An LLM scraping a typical SaaS docs site might ingest three paragraphs of actual API detail buried inside ten paragraphs of sales copy, repeated headers, and footer links.

llms.txt is a proposed convention — similar in spirit to robots.txt or sitemap.xml — that gives site owners a dedicated, clean file where they can tell AI systems exactly what their site contains and where the best machine-readable content lives.

What the File Actually Looks Like

The format is intentionally simple: a Markdown file placed at https://yourdomain.com/llms.txt. It typically contains:

A short description of what the site or product is
A structured list of links to key pages, particularly Markdown or plain-text versions of documentation
Optional metadata about the project's purpose, license, or intended audience

# Acme API Docs

> Acme provides a REST API for payment processing in West Africa.

## Docs
- [Authentication](https://acme.dev/docs/auth.md): How to obtain and rotate API keys
- [Payments](https://acme.dev/docs/payments.md): Create, capture, and refund transactions
- [Webhooks](https://acme.dev/docs/webhooks.md): Event payloads and retry logic

## About
- [Changelog](https://acme.dev/changelog.md): Version history

The companion convention llms-full.txt takes this further — it is a single concatenated file containing the entire documentation corpus, ready for an LLM to consume in one shot without making multiple HTTP requests.

Why This Is More Than a Neat Trick

At first glance this looks like a minor developer convenience. Look closer and it is a meaningful shift in how software interfaces with software.

AI agents are becoming primary API consumers. Tools like Devin, OpenHands, and Cursor's agent mode do not just assist humans — they autonomously read documentation, write integration code, and execute it. If your docs are hard for an agent to parse, your API is effectively harder to adopt. llms.txt is, in a real sense, a developer experience investment.

Training data quality matters downstream. Model providers and fine-tuning pipelines scrape the web constantly. A well-structured llms.txt with links to authoritative, accurate documentation increases the probability that future models learn correct information about your product rather than hallucinating API signatures from half-scraped HTML.

It reframes who your documentation is written for. For the past decade, "write good docs" meant optimising for human readability, search engine indexing, and visual hierarchy. Now there is a third reader in the room that does not care about your colour scheme but very much cares about semantic structure and completeness.

Adoption and the Ecosystem Around It

Several notable projects and documentation platforms have begun shipping llms.txt files. The pattern is being formalised by a small but active community, and tools are appearing that auto-generate the file from existing documentation pipelines — whether that is a Docusaurus site, a GitBook instance, or a hand-rolled Markdown repo.

The convention is not yet an official standard ratified by any body. That is fine. robots.txt was never formally standardised either, and it became universal because it solved a real, widely felt problem. The same dynamics are at play here.

For SaaS founders and engineering teams, the practical calculus is straightforward: the effort to produce an llms.txt file is low — a few hours at most for a mature product — and the compounding upside as AI-assisted development grows is significant.

What Good Implementation Looks Like

A few principles worth following if you adopt this convention:

Link to Markdown, not HTML. If your docs platform can serve .md versions of pages, use those URLs. Raw Markdown is dramatically easier for LLMs to parse than rendered HTML.
Be accurate, not exhaustive. An llms.txt that links to 200 pages of outdated content is worse than one that links to 20 pages of current, correct documentation.
Version it. Treat the file like code. When your API changes, the file should change too. A stale llms.txt pointing to deprecated endpoints will cause real problems for developers relying on AI-assisted integration.
Include a brief product description. Even two sentences of context help a model orient itself before diving into technical references.

The Broader Signal

This convention is a symptom of something larger: the web's infrastructure is quietly being retrofitted for machine readability. Just as RSS once made content portable across readers, llms.txt is part of a pattern where structured, unambiguous content wins over richly designed but semantically noisy pages.

The developers who build this infrastructure today — clean APIs, well-structured docs, machine-readable metadata — are positioning their products to be the ones that AI agents reach for first when writing integration code tomorrow.

Source: Anna's Archive Blog — https://annas-archive.gl/blog/llms-txt.html

Why this matters for your project: Whether you are shipping a public API, a developer tool, or an internal platform, adding an llms.txt file is one of the lowest-effort, highest-leverage documentation improvements you can make right now. At Code!nk, we factor this kind of AI-readiness into the products we build — because the developers using your software increasingly have an AI agent in the loop, and that agent deserves a clean front door.