llms.txt and Google Search: Know What the File Does—and What It Doesn’t

An llms.txt file is often presented as a required shortcut for getting cited by AI. That claim goes further than Google's published guidance supports. Google's May 2026 guide says there are no additional technical requirements or special AI text files needed for appearance in Google Search's AI Overviews or AI Mode; it also calls out unnecessary AI text files such as llms.txt among tactics site owners can ignore for Google Search. This is a Google Search statement. It does not establish what every other AI product reads or ignores. Google: AI features and your website · Google's guide to generative AI features

So decide what you need the file to do. If a documentation platform, tool, or internal workflow has a documented requirement for a text index, evaluate that requirement on its own. If the goal is Google visibility, prioritize crawlable, indexable, useful pages and Search Console evidence; adding llms.txt is not a substitute for those foundations and is not a ranking promise.

Separate the file's possible uses

The same filename can be used for different purposes, so first name the owner and intended reader:

Intended use	Question to answer	Evidence of value
Documentation navigation	Does a specific tool or documentation stack read the file?	The tool's current official documentation and a successful parse or user task
Human-maintained content map	Does the file help a person find key docs?	Readers use it, and links stay accurate as pages change
Google Search discovery	Does Google require this file to crawl or show pages?	Google's current published Search guidance; currently it says no AI text file is required
AI citation or traffic	Does the file cause another AI service to cite or send traffic?	A controlled, service-specific observation; the existence of the file alone is not evidence

Do not merge these questions into “AI discoverability.” A navigation artifact can be useful to people while having no documented Google ranking role. A crawler request can show that a bot fetched a URL; it does not show that a model used the file, cited a page, or sent a visitor.

For Google, start with pages and access

Google's documented eligibility path for AI Search features uses the existing Search requirements: a page must be indexed and eligible to appear with a snippet. Google says no additional technical requirements apply to AI Overviews or AI Mode, and it emphasizes accessible crawling, internal links, useful content, text availability, page experience, and markup that matches visible content. It also states that meeting requirements does not guarantee crawling, indexing, or serving. Google Search technical requirements

When a page is missing, inspect the page itself: status code, robots directives, canonical URL, rendered content, internal links, and Search Console reports. A text index file does not repair a blocked page, a duplicate URL, a thin answer, or a broken route. Do not change robots.txt or crawler access to compensate for a missing optional file.

Google reports AI-feature traffic in Search Console's Web search type, according to its current guidance. Use available Search Console and Analytics data to distinguish search impressions, clicks, sessions, and conversions. Do not label a request to /llms.txt as an AI citation, a referral, or a customer visit.

If you keep the file for people or a documented tool

Keep it small, accurate, and low-maintenance:

State what the file is for and who maintains it.
Link to stable, canonical public pages that actually exist.
Group links by a reader task, such as setup, reference, or troubleshooting.
Remove private, draft, expired, or redirecting URLs.
Compare its links with the sitemap or canonical page list after major releases.
Record which tool or audience uses it; if no one depends on it, reconsider maintenance effort.

Avoid copying full site pages into the file unless a documented consumer expects that format. A list that drifts out of date can mislead its intended reader. Do not add claims such as “preferred by ChatGPT” or “increases AI citations” without evidence from the named service.

Measure what the file can and cannot tell you

If you inspect server logs, separate user-agent requests, status codes, bytes served, and the requested path. Verify crawler identity using the relevant service's published method when one exists. A user-agent string by itself can be spoofed. Then ask a separate question: did the request lead to a measurable referral, citation, or task completion? That requires service-specific referral or citation data, not just a file hit.

For a small site, a simple change record can be enough: purpose, links added or removed, date, tool documentation reviewed, and observed user outcome. Do not build an expensive monitoring system merely to watch one optional text file. If you compare before and after, keep page changes, source changes, and timeframe visible so the file does not get credit for unrelated changes.

llms.txt decision card

Goal: [documentation navigation / explicit integration / other]
Named consumer: [tool or reader]
Official requirement or evidence: [URL / none found]
Canonical links included: [count and review date]
Private or obsolete links excluded: [check]
Search Console status of priority pages: [evidence]
Log interpretation boundary: [request / no citation conclusion]
Maintenance owner and review trigger: [person / event]
Decision: [keep / simplify / remove from workflow] — reason: [documented use]

llms.txt references and limits

Google's guidance describes Google Search. It does not establish a universal policy for all AI assistants, developer tools, or documentation systems. Check the official documentation for the specific consumer before attributing any effect to llms.txt.

Short-video script: Does llms.txt help Google Search?

Hook: “If someone says llms.txt is required for Google AI citations, ask which Google requirement they mean.”

Beat 1: “Google's current Search guidance says no additional AI text file is required for AI Overviews or AI Mode.”

Beat 2: “A docs tool may still have its own documented file format. That is a separate use case.”

Beat 3: “A bot request is only a fetch. It does not prove a citation, ranking gain, or customer visit.”

Close: “Name the consumer, check its docs, and spend effort on useful pages and crawl access first.”

Educational carousel: Four questions before maintaining an AI text file

Cover: llms.txt: documented use or visibility folklore?
Who reads it? Name the tool, person, or workflow.
What does its documentation say? Confirm the current requirement at the source.
What does Google say? No extra AI text file is required for its AI Search features.
What does a log hit prove? A request happened; citation and referral remain separate questions.
Can you maintain the links? Keep only stable, public, useful pages.
Decision: Keep it for a real consumer, not as a ranking guarantee.

Field experiment thread: Evaluate a file without confusing correlation and cause

1/ Write the intended consumer and task. Capture its official documentation or a real user requirement before changing the file.

2/ Record the baseline: current file, links, site release changes, Search Console visibility for priority pages, and any verified referral path.

3/ If a real consumer exists, make one bounded file change and verify that the consumer can parse the expected links. Do not alter robots or canonical settings as part of this test.

4/ Observe only a metric the consumer can actually affect, such as successful documentation navigation. Keep requests, citations, referrals, and conversions as separate outcomes.

5/ Retain the file only if maintenance effort serves a documented user or system need. A change in search traffic by itself does not prove that the file caused it.

Tool-neutral prompt pack: Review an llms.txt proposal

Prompt 1 — Consumer check

Review this llms.txt proposal [file] for the named consumer [tool/person]. Using only the supplied official documentation [links], determine whether the format is required, optional, or unsupported. Do not infer benefits for unnamed AI systems.

Prompt 2 — Link hygiene

Check these proposed links [list] against the canonical URL inventory [source]. Flag private, missing, redirecting, duplicate, or obsolete pages. Do not add pages that are not in the inventory.

Prompt 3 — Evidence interpretation

Interpret these logs and referral records [data]. Separate verified fetches, referrals, citations, and conversions. Flag user-agent identity uncertainty and missing evidence. Do not treat a request for /llms.txt as proof that a model used it.

Stop-doing swipe file: AI-file promises

Stop: “Publish llms.txt and ChatGPT will cite you.”
Use: “Which service documents this file, and what measurable outcome would show value?”

Stop: “The bot fetched the file, so it recommended the site.”
Use: “The server log records a request. Citation and referral need separate evidence.”

Stop: “Google requires this file for AI Overviews.”
Use: “Google's current guidance says no additional AI text file is required for those Search features.”

Stop: “A file replaces crawlability and useful pages.”
Use: “Check the page's access, index eligibility, content, and internal links directly.”

Hook–Value–Template: Keep only a file with a named job

Hook: “An optional file is worth maintaining only when someone or something actually uses it.”

Value: Separate product documentation, human navigation, Google Search requirements, and AI referral measurement. This keeps a simple text artifact from becoming a false promise about rankings or citations.

Template: “We maintain llms.txt for [named consumer] to help it [specific task]. The current requirement is documented at [source]. The file links to [canonical pages] and is reviewed by [owner] after [trigger]. We measure [observable outcome]. We do not infer [citation/ranking/referral] from a file request alone.”

Example: “We keep a short docs index for our internal assistant because our team uses it to locate public setup guides. Google Search does not require the file for AI Overviews or AI Mode, so we do not count its existence as an SEO result.”

Keep the file only for a specific, verifiable job; do not sell it as a visibility switch.