HOW TO CREATE LLMS TXT

How to Create an llms.txt File

The llms.txt file is an emerging standard designed to provide Large Language Models (LLMs) and AI agents with a concise, high-context map of a website's documentation. Much like robots.txt tells crawlers what not to index, llms.txt tells AI exactly what it should read to understand your project.

1. Core Principles

• Location: Place the file in the root directory of your website (e.g., https://example.com/llms.txt).

• Format: Use standard Markdown.

• Goal: Provide a summary of the project and a curated list of links to the most important documentation pages.

2. File Structure

An llms.txt file typically consists of four main sections:

A. The H1 Header

The file should start with a single # header containing the name of the project or site.

B. Brief Summary

Immediately following the H1, provide a short paragraph (2-3 sentences) explaining what the project is. This helps the LLM decide if the resource is relevant to the user's query.

C. Optional Details

You can include a blockquote or a short list of key metadata (license, version, primary tech stack).

D. The Link Sections (H2 Headers)

Organize your links under H2 headers. These headers usually group content by "Core Documentation," "Tutorials," "API Reference," etc.

• Links must be in a Markdown list.

• Each link should be followed by a short, descriptive sentence explaining what that specific page covers.

3. Creating "Full" vs. "Small" context

There is an optional extension where you provide a more detailed version at /llms-full.txt.

• llms.txt: A concise map of the site.

• llms-full.txt: A single file containing the actual content of all the documentation, concatenated for easy ingestion.

4. Best Practices

• Use Absolute URLs: Always use full URLs (https://...) so the LLM doesn't have to guess the base path.

• Be Concise: Don't include marketing fluff. LLMs prioritize token efficiency.

• Link to Markdown: If your site has raw Markdown versions of pages, link to those instead of the HTML versions to save tokens.

• Hierarchy: Put the most important "Getting Started" or "Architecture" links at the top.

5. Implementation Example

See the companion file example_llms_txt.md for a ready-to-use template.