More people now ask an AI assistant the questions they used to type into a search box: which salons near me are good with curly hair, who does lived-in blonde, where can I get a keratin treatment this weekend. An assistant can only mention your salon accurately if it has been able to read something about you.
For many salons the problem is not that the website is bad. It is that the website quietly tells these assistants to stay away.
How a website says "do not read me"
Every website can have a small text file at /robots.txt that tells automated visitors, called crawlers, which parts of the site they may read. Google explains how it works in its introduction to robots.txt.
Each crawler has a name. A few of the ones that matter here:
- GPTBot, OAI-SearchBot and ChatGPT-User belong to OpenAI, which describes what each one is for on its page about its crawlers. OpenAI says sites that block OAI-SearchBot are not shown in ChatGPT's search answers.
- ClaudeBot belongs to Anthropic.
- PerplexityBot belongs to Perplexity.
- Google-Extended controls whether Google may use your content for its Gemini models. It does not affect Google Search, which uses Googlebot. Google lists all of its crawlers in its overview of Google crawlers.
A robots.txt file with a group for one of these names that says "Disallow: /" shuts that crawler out of the whole site.
How salons end up blocked without deciding to
Almost nobody sets out to hide their salon. It usually happens as a side effect:
- a security or SEO plugin with an AI-blocking option switched on
- a hosting or network service that offers, or defaults to, blocking AI crawlers
- a robots.txt copied from another site, or written by a previous developer
- a site builder setting that was changed years ago and forgotten
There can be good reasons to block some of these crawlers, for example if you do not want your writing used to train models. The point is that it should be a choice you made. For a local business that wants to be recommended, blocking every assistant is usually the wrong one.
How to check your own site
Type your web address followed by /robots.txt into a browser, for example yoursalon.com/robots.txt. If you see a line naming one of the crawlers above, followed by a line that says Disallow with only a slash after it, that crawler is blocked.
Reading these files by eye is fiddly, because each rule applies to the group of names above it. Our free website check reads your robots.txt the way the crawlers do, and tells you plainly which assistants, if any, are shut out. It also looks for an llms.txt file and a few other signals that help assistants describe you.
What llms.txt is
llms.txt is a proposed convention: a plain text file at /llms.txt that describes, in simple markdown, what a website is and where its important pages are. It is not an official standard, and not every assistant reads it yet. It is cheap to add, it does no harm, and it gives an assistant a clean summary instead of whatever it can piece together from your pages.
For a salon, a useful llms.txt says who you are, where you are, what you offer and roughly what it costs, and links to your services, team and contact pages. Every salon website on Gloss Front serves its own, generated from the salon's published details.
Being worth recommending
Being readable is only the first step. An assistant is more likely to describe your salon well when your website answers the questions people actually ask, in your own words: what a service involves, how long it takes, what it costs, who does it and how to book. That is the same content that helps with search, and we go into it in how to write salon blog posts that people actually find.
Check the file first. It takes a minute, and if something is blocked, the fix is usually one line.