Roughly one in ten websites now publishes a file called llms.txt, sold as the way to make AI assistants read your site properly. Server logs tell a less flattering story: the AI crawlers that file was written for mostly never open it.
That gap between what the file promises and what it currently delivers is worth understanding before you pay anyone to add one. Here is what llms.txt is, who honours it, and what actually moves the needle instead.
What Is llms.txt?
llms.txt is a plain-text file in Markdown format placed at the root of your website, at yoursite.com/llms.txt. It gives AI systems a short, curated summary of what your site is and links to the pages that matter most. Think of it as a hand-written map for machines, rather than letting them wander.
The idea addresses a real problem. A modern web page is mostly navigation, cookie banners, scripts and layout. The actual answer to a customer's question might be four sentences buried in the middle. llms.txt was proposed as a way to hand over the four sentences directly.
The specification is published at llmstxt.org. It is a community proposal, not a standard issued by any search or AI company, and that distinction explains most of what follows.
How Is llms.txt Different From robots.txt?
robots.txt controls permission. llms.txt offers guidance. One tells a crawler where it may not go, the other suggests what is worth reading. They solve opposite halves of the same problem and neither replaces the other.
The practical difference
--- robots.txt is a long-established convention that every serious crawler checks. It is a list of rules about access.
--- llms.txt is a newer proposal that some AI systems check. It is a curated summary about meaning.
For a Hong Kong business owner the ranking of priorities is unambiguous. If your robots.txt accidentally blocks AI crawlers, no llms.txt file on earth will help you, because the machine never gets far enough to read it. Check the permission file first. Blocked crawler access is a genuinely common and genuinely fatal problem, and it is invisible from the front end of your website.
Do AI Assistants Actually Read llms.txt?
Support is partial and honestly stated by the vendors themselves. Anthropic has publicly confirmed support. Perplexity has said it retrieves llms.txt to help prioritise which pages to select. OpenAI has not officially adopted the standard. Google has declined on the record.
Where each major system stands
--- Claude, from Anthropic: support publicly confirmed.
--- Perplexity: retrieves the file and uses it to prioritise page selection.
--- ChatGPT, from OpenAI: no official adoption. Some teams report correlated changes in citation patterns, which is not the same as confirmation.
--- Google: declined publicly. In July 2025 Gary Illyes confirmed Google does not support llms.txt and is not planning to, and John Mueller compared the idea to the long-discredited keywords meta tag.
Then there is the log-file evidence, which is where enthusiasm meets reality. Analyses of real crawler traffic find that GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot and Google-Extended overwhelmingly skip /llms.txt and crawl the HTML directly. Multiple studies conclude that publishing the file does not measurably improve your odds of being cited by ChatGPT, Claude, Gemini or Perplexity today.
Adoption numbers show the same split between talk and traction. A study of 300,000 domains by SE Ranking found a 10.13% adoption rate. A May 2026 crawl of the Tranco top 10,000 sites found only about 5.86% had a valid llms.txt, and adoption was concentrated in developer-tool and documentation sites rather than ordinary businesses.
What Goes Inside an llms.txt File?
The format is deliberately simple: Markdown, human-readable, no code. The specification requires an H1 heading with your site or project name, then a blockquote holding a one-paragraph summary. Everything after that is optional free-form Markdown with links.
The required shape
--- An H1 line with the name of your business or site. This is the only strictly required section.
--- A blockquote summary of one paragraph, containing the key facts needed to understand everything else.
--- Optional sections of ordinary Markdown with links to your most important pages, each with a short description.
For a Hong Kong SME, a useful version answers what you sell, where you operate, who you serve, your hours, and where the answers live. A restaurant group's file would name the districts, the cuisines, the booking page and the opening hours. A trading firm's would name the product categories, the markets served and the minimum order terms.
The whole exercise is a thirty-minute job for someone who can edit a text file and upload it. That cost matters to the verdict below.
What Does a Good llms.txt Look Like for a Hong Kong SME?
A good one reads like a briefing note for a new staff member who has thirty seconds. Name the business, state what it does and where, then point to the five or six pages that carry the commercial answers. Vague marketing language is the most common mistake.
Take a Hong Kong air-conditioning maintenance company as the worked example. The weak version of its summary reads: "We are a leading provider of innovative HVAC solutions committed to excellence." That sentence tells a machine nothing a customer could use.
The useful version reads: "Air-conditioning installation and maintenance for commercial premises in Hong Kong, serving Kowloon and Hong Kong Island. Split-type and VRV systems. Emergency call-out within four hours. Contracts from twelve months. Established 2004."
Every clause in the second version is a fact an AI assistant can repeat when someone asks for an air-conditioning contractor in Kowloon with emergency service. That is the whole purpose of the exercise.
Four steps to write yours
--- List your five commercial pages. Services, pricing or quotation, coverage area, contact, and the one page that answers your most-asked question.
--- Write the summary paragraph in facts, not adjectives. What you sell, where, to whom, on what terms, since when.
--- Add one line per link saying what a reader will find there, in plain language.
--- Save as llms.txt and upload to your web root, then open yoursite.com/llms.txt in a browser to confirm it loads as plain text rather than downloading as a file.
Two mistakes worth avoiding. Do not paste your entire website into it, because the value is selection. And do not include anything you would not want quoted verbatim in an AI answer to a stranger, because that is precisely what the file invites.
Should a Hong Kong Small Business Bother?
Yes, but only in that order of operations. It costs thirty minutes, it carries no downside, one major assistant confirms support and another actively uses it. What it must not do is absorb the budget or attention that belongs to the things AI systems demonstrably do read.
Write one if
--- You can do it yourself or it is bundled into work you are already paying for.
--- Your site has a small number of pages that carry most of the commercial value.
--- Your structural basics are already sound.
Do not pay a premium for it if
--- Someone is selling it as the reason AI does not mention your business.
--- Your website is still blocking AI crawlers, missing structured data, or hiding prices and contact details inside images.
--- The quote is priced as a project rather than a task.
The uncomfortable framing: a beautifully written llms.txt on a site AI cannot otherwise read is a polite note pinned to a locked door.
What Actually Makes AI Cite Your Business Instead?
AI systems mostly read the same HTML that search engines read, so the levers that work are unglamorous and well understood. Four of them account for most of the difference between a business AI can quote and one it cannot.
Crawler access
Confirm your robots.txt does not block GPTBot, ClaudeBot, PerplexityBot or the other named AI agents. This single check has decided more AI visibility outcomes than any file you can add.
Structured data
JSON-LD markup describing your organisation, your location, your products and your frequently asked questions gives machines unambiguous facts rather than paragraphs to interpret. Incomplete schema is one of the most common findings in AI readability audits.
Extractable answers
Put the answer immediately under the question, in forty to sixty words, in plain text. A page that buries its conclusion in the eighth paragraph is hard to quote, and being quotable is the entire game.
Corroboration elsewhere
AI systems weigh whether anyone else confirms what you claim. Mentions on directories, industry sites, forums and reputable local media do work your own website cannot do for itself.
None of that is exotic. It is why a structural audit tends to surface bigger wins than a new file.
Frequently Asked Questions
Will llms.txt hurt my Google rankings?
No. Google has said it does not use the file, and an unused file is not a penalty. It is neutral for search, which is different from being useful.
Where exactly does the file go?
At the root of your domain, reachable at yoursite.com/llms.txt. The specification permits a subpath, but root placement is what tools look for.
Do I need a developer to add it?
Usually not. If you can upload a file to your web host or add one through your content management system, you can publish it. The writing is the harder half.
How often should I update it?
Whenever the facts change: new services, new locations, revised hours, a new key page. A stale summary is worse than none, because it invites AI to state something outdated about you with confidence.
Is llms.txt the same as an AI sitemap?
Not quite. A sitemap lists every URL for completeness. llms.txt deliberately selects a few and explains why they matter. Curation is the point.
If crawlers ignore it, why has adoption reached 10%?
Partly because it is cheap and harmless, partly because documentation-heavy sites genuinely benefit, and partly because agencies can bill for it. All three can be true at once.
The Takeaway
llms.txt is a sensible idea with thin adoption and partial support. Claude confirms it, Perplexity uses it, OpenAI has not committed and Google has said no. Real crawler logs show the file is mostly skipped.
So write one, spend half an hour on it, and then spend the rest of the week on crawler access, structured data and answers a machine can lift cleanly off the page. That order is what changes whether AI mentions your business.
Being early to a standard matters far less than being readable today. Twenty-eight years of Hong Kong technology cycles has taught us which is which. We understand AI. UD stands with you.
Reviewed by the UD AI team, Hong Kong. Published 30 July 2026.
Want to Know What AI Can and Cannot Read on Your Site?
A free 30-second scan checks your site against 170+ AEO structure checks, covering crawler access, JSON-LD coverage and how quotable your pages are, then returns a prioritised action plan. No sign-up, no credit card. We will walk you through every finding step by step and tell you what to fix first.
