llms.txt

Learn what llms.txt and llms-full.txt are, why agents use them, and how to verify exports on your published site.

When an AI tool needs to answer questions about your product, it first needs to know what documentation exists and where to read it. Sidebars and HTML search work for humans; agents benefit from a compact, machine-readable index they can fetch in one HTTP request.

Before you begin

  • A published site on a public GitHub repository (see Agent-ready docs)
  • Your {owner} and {repo} slugs from the production URL

Overview

Every public docs.page site exposes two auto-generated markdown exports that follow the llms.txt convention:

  • llms.txt: a lightweight index with site title, description, and a Docs section listing each page as a markdown link with its title, summary, and canonical URL
  • llms-full.txt: the same header, then full MDX source for every page (title, Source: URL, and raw content in a fenced block)

Neither export requires a docs.json toggle or build step. docs.page generates both from the same GitHub source as your live site. Push an update and the exports reflect it on the next request.

Production URLs:

text
https://docs.page/{owner}/{repo}/llms.txt
https://docs.page/{owner}/{repo}/llms-full.txt

Branch and pull request previews use the same ~ref segment as page URLs. Custom domains and vanity subdomains serve the same paths on your public origin. See llms.txt on Features for generation rules, caching, and URL variants.

How it works

What agents get from llms.txt

llms.txt is a map of your docs (titles, summaries, and canonical links) without downloading full page source. Example shape:

markdown
# Acme Handbook

Product documentation for the Acme platform.

## Docs

- [Getting started](https://docs.page/acme/handbook/getting-started): Install the SDK and make your first API call.
- [Authentication](https://docs.page/acme/handbook/authentication): API keys, OAuth, and token refresh.

Agents and integrators often add this URL to a system prompt, tool manifest, or scheduled ingestion job. The tool fetches the index first, then follows links to pages relevant to the user's question.

What agents get from llms-full.txt

llms-full.txt bundles full MDX source for every page in one response. Use it when an agent cannot follow links or prefers a single bulk download over many HTTP requests.

The tradeoff is size: large sites produce large responses. Prefer llms.txt when discovery alone is enough.

Agent needUse
Map of pages: titles, summaries, linksllms.txt
Full source in one downloadllms-full.txt
Per-page reads after discoveryllms.txt, then follow links, or use MCP server
One page's raw source in the browserAppend .md to the page URL, or choose View markdown in the page action menu

How agents use exports in practice

Typical workflows:

  1. Prompt context: include the llms.txt URL so the model knows where to find your catalog
  2. Scheduled ingestion: a job fetches llms.txt or llms-full.txt and loads content into a vector store
  3. On-demand fetch: an agent curls the index at the start of a session, then retrieves individual pages by URL

RAG pipelines parse the Docs links from llms.txt and fetch pages selectively. One-shot pipelines ingest llms-full.txt when a single markdown blob is simpler than many requests.

Check yours

bash
curl -sS "https://docs.page/{owner}/{repo}/llms.txt" | head

Expect markdown with a Docs section listing your pages.

For branch previews, add ~{ref} after the repository name (the same rules as branch preview):

bash
curl -sS "https://docs.page/{owner}/{repo}~feature-branch/llms.txt" | head

For very large repositories, the response may include an x-docs-page-tree-truncated: 1 header, meaning the export can be incomplete. Use MCP server to read individual pages instead.

Check one page

To verify raw source for a single page without fetching the full site index, open View markdown from the page header menu, or append .md to the page URL:

text
https://docs.page/{owner}/{repo}/getting-started.md

See HTTP endpoints: Raw markdown for path rules.

Related