Skip to content

    llms.txt What Is It and How to Write One for Your Site

    Stephens Quantum TeamNovember 20, 20267 min read

    llms.txt is a markdown file that gives AI agents clean site content. Learn its purpose, why models read it, and the steps to create and host one so pages can be cited in answers.

    Small business sites often see AI answer engines pull content without clear signals on what to use. llms.txt addresses this by giving models a direct path to preferred information.

    What llms.txt actually is

    The file is a markdown document named llms.txt. It sits at the site root or any subpath and covers pages under that path [1]. It starts with an H1 title for the site or project, followed by a blockquote summary. Additional sections can include paragraphs or lists, then H2 headers that introduce lists of links to further markdown files [1].

    This structure stays small enough to fit in an agent's context window while pointing to detailed content only when needed. The file can reference markdown versions of pages created by appending .md to the original URL [1].

    Why AI answer engines read llms.txt

    Agents fetch websites constantly to answer questions or complete tasks. HTML pages wrap information in navigation and scripts, making clean extraction difficult [1]. Context windows limit how much content fits, so agents prefer concise expert summaries in one location.

    llms.txt provides that overview in a format language models understand. Agents view or search the file first, then follow links to the actual details. AI labs already publish their own llms.txt files for developer documentation, showing the approach has become standard [1].

    How llms.txt works with robots.txt

    Robots.txt controls which crawlers may access paths on a site [2]. llms.txt instead supplies context for allowed content when an agent needs topic information. The two files serve different purposes and can exist together without conflict [1].

    A site can keep its existing robots.txt rules while adding llms.txt for AI discovery. This separation means no changes to robots.txt are required.

    Steps to create and host an llms.txt file

    Start by writing the file in any text editor that saves plain UTF-8 text. Begin with the site name as an H1, add a short summary in a blockquote, then include any needed details in regular markdown sections. Use H2 headers to group links to markdown files or other resources [1].

    Place the finished file at the root path such as https://example.com/llms.txt or inside a subfolder like /docs/llms.txt to cover only those pages. Upload it through the site's hosting control panel or file manager so it is publicly accessible.

    After upload, confirm the file loads in a browser at its exact URL. Agents will locate it automatically when they check the domain.

    Linking pages so they can be cited

    Add HTML link elements or HTTP headers that point agents to the llms.txt file and to markdown versions of pages. The relation rel="describedby" points to the llms.txt file while rel="alternate" type="text/markdown" points to clean page versions [1].

    These links help agents find the curated content. A small business site owner can add the header once at the server or CDN level without editing every page.

    See What is local SEO for small business sites for how discovery files fit into broader visibility efforts. The How to Read the Search Console Page Indexing Report explains how to check whether pages remain reachable after adding new files.

    Sources & citations

    Every figure or external claim above links to a primary source. No fabricated statistics.

    1. [1]llms.txt (llmstxt.org), llmstxt.org. Accessed 2026-11-20.
    2. [2]Robots Exclusion Protocol (IETF), IETF. Accessed 2026-11-20.

    Stephens Quantum Team

    Stephens Quantum LLC builds custom CRMs, e-commerce systems, SEO-first web apps and Beacon, a first-party attribution tool.