How to Create an llms.txt File: A Practical Implementation Guide

This guide walks through writing, hosting, and maintaining an llms.txt file for your website. For a short definition of the term, see our llms.txt glossary entry. Here we focus on the practical work.

Set expectations first: llms.txt is a community proposal, not an adopted standard. No search engine or AI provider is required to read it, and support varies by tool. It is not a ranking factor, and it does not guarantee that any crawler will fetch it or that any AI system will cite your content. Treat it as inexpensive documentation hygiene, not a growth tactic.

What you are creating

An llms.txt file is a Markdown document served at /llms.txt on your domain root. It gives readers, human or machine, a short orientation to your site and a curated list of links to your most useful pages, so they do not have to dig through navigation menus, scripts, and page chrome to find them.

The proposal was published by Jeremy Howard of Answer.AI, and the current text lives at llmstxt.org. Check it before you start, because proposals evolve.

When it makes sense

Consider an llms.txt file if:

• You maintain documentation: Developer docs, API references, or help centers with many pages benefit most from a curated index.
• You can keep it current: A stale file points readers to outdated pages, which is worse than no file.
• You have a clear idea of your key pages: The value comes from curation, so you need to be able to decide which pages matter. It is probably not worth the effort for a small site with a handful of pages, or if you have no one responsible for maintaining it.

File structure

The proposal describes a simple Markdown layout:

1. An H1 title: The name of the project or site. This is the only required element.
2. A short blockquote summary: One to a few sentences describing what the site or project is.
3. Optional prose sections: Paragraphs or lists with any extra context a reader needs. These should not contain headings.
4. H2 sections of links: Each section groups links in the form - [Page title](url): short description.
5. An optional ## Optional section: By convention in the proposal, links here are secondary and can be skipped when a shorter overview is needed.

Template

# Project Name
 
> One or two sentences describing what the project does and who it is for.
 
Any important background a reader should know, such as supported versions or where to find the changelog.
 
## Documentation
 
- [Getting started](https://example.com/docs/start): Install and make your first request.
- [API reference](https://example.com/docs/api): Endpoints, parameters, and response formats.
- [Authentication](https://example.com/docs/auth): How to create and rotate API keys.
 
## Guides
 
- [Rate limits](https://example.com/docs/limits): Request quotas and how to handle limit errors.
- [Error handling](https://example.com/docs/errors): Error codes and recommended responses.
 
## Optional
 
- [Changelog](https://example.com/changelog): Release history.
- [Community forum](https://example.com/community): Questions and discussion.

Step-by-step setup

1. Decide which pages matter: List the pages a new reader most needs. Aim for quality over quantity.
2. Write the header: Add your H1 title and a short, factual summary.
3. Group your links: Organize links under H2 headings that match how readers think about your content.
4. Describe each link: Add a short description after each link that says what the page covers. Avoid marketing language.
5. Move secondary links to ## Optional: Anything useful but not essential goes here.
6. Save as UTF-8 Markdown: Name the file llms.txt with no extra extension.
7. Publish at your domain root: Make it available at https://yourdomain.com/llms.txt over HTTPS.
8. Validate it: Follow the checklist below before announcing or relying on it.

Examples

Minimal

# Acme API
 
> A payments API for online businesses.
 
## Documentation
 
- [Getting started](https://acme.dev/docs/start): Quickstart guide.
- [API reference](https://acme.dev/docs/spec): Endpoint reference.

A short file like this suits a small project with a handful of key pages.

Well-curated

# Cloud Platform Docs
 
> Hosting for serverless functions, static assets, and edge databases.
 
Documentation covers the current major version. Older versions are archived.
 
## Essential guides
 
- [Deployment quickstart](https://cloud.example/docs/deploy): CLI deployment steps.
- [Environment variables](https://cloud.example/docs/env): Managing secrets across environments.
 
## Limits and pricing
 
- [Rate limits](https://cloud.example/docs/limits): Request quotas and concurrency limits.
- [Pricing model](https://cloud.example/docs/pricing): How usage is billed.
 
## Optional
 
- [Changelog](https://cloud.example/changelog): Release history.

This version groups links by purpose, describes each one, and sets aside secondary material.

What to avoid

# Best Cheap Cloud API Provider Free Fast Tools
 
> The best affordable cloud API provider with top performance.
 
## Links
 
- [Link 1](https://example.com/page1)
- [Link 2](https://example.com/page2)
... (hundreds of uncurated URLs with no descriptions)

Keyword-stuffed titles, promotional claims, long uncurated link dumps, and missing descriptions all defeat the purpose of the file.

Hosting and serving the file

• Location: Serve it at the root of your domain, for example https://yourdomain.com/llms.txt.
• Access: Keep it publicly reachable, without logins, cookies, or redirect loops.
• Content type: Serve it as text/plain or text/markdown with UTF-8 encoding.
• HTTPS: Serve it over HTTPS.
• Robots rules: Make sure your robots.txt does not accidentally block the file if you want it to be reachable.

Validation checklist

Before relying on the file, confirm each of the following:

• The file loads: Request /llms.txt in a browser and with a command-line tool, and confirm you get a 200 response and the expected Content-Type.
• The structure is valid: There is exactly one H1, and links are formatted as Markdown links with short descriptions.
• Every link works: Each URL returns a 200 response and does not redirect through a login page.
• Links point to canonical pages: Avoid tracking parameters and outdated URLs.
• The summary is accurate: It matches what the site currently offers.
• No sensitive content: The file is public, so do not include internal or private URLs.

Keeping it current

A stale llms.txt file sends readers to outdated pages. To avoid that:

• Assign an owner: Make one person or team responsible for the file.
• Add it to your release checklist: Review the file whenever you add, rename, or remove major documentation pages.
• Check links on a schedule: Run a periodic link check so broken URLs are caught early.
• Keep summaries factual: A summary that overstates your product can mislead readers, and an out-of-date one can mislead them just as much.

What to expect

• Support varies: Some tools and documentation platforms read or generate llms.txt files, and many do not. Check the documentation of the specific tools you care about rather than assuming support.
• No guaranteed crawling: Publishing the file does not oblige any crawler or assistant to fetch it.
• No guaranteed citation or ranking effect: There is no reliable evidence you should count on that the file changes search rankings or how AI tools cite your pages.
• A summary can be incomplete: Short summaries leave out nuance, so keep your actual pages clear and complete rather than relying on the file to do that work.

How it relates to robots.txt and sitemap.xml

llms.txt does not control access. If you want to allow or restrict particular crawlers, use robots.txt and your server's access controls.

Frequently asked questions

Is llms.txt a standard?

No. It is a community proposal. It is not a formal internet standard, and no search engine or AI provider is required to support it.

Is llms.txt mandatory?

No. Websites work normally without it. Adding one is optional.

Does llms.txt improve SEO or AI citations?

You should not count on it. We are not aware of reliable evidence that it changes search rankings or guarantees that AI tools will cite your content.

What is the difference between llms.txt and robots.txt?

robots.txt tells crawlers which paths they may request. llms.txt is a curated guide to your most useful pages and does not grant or restrict access.

What is llms-full.txt?

Some documentation tools also generate a file called llms-full.txt that combines page content into one large Markdown file. It is a convention used by some tools, not part of the core proposal, and support varies.

Where do I put the file?

At the root of your domain, so it resolves at https://yourdomain.com/llms.txt.

Do I still need a sitemap?

Yes, if you want search engines to discover your pages efficiently. The two files serve different purposes.

Summary

An llms.txt file is a low-cost way to give readers a curated entry point to your documentation. Keep it short, factual, and current, validate it before you publish, and keep your expectations modest. If you also want clean text from your pages for other uses, see our guide on converting web pages to Markdown.