You cannot opt out of AI by editing your robots.txt, and this week Google said so. Cloudflare’s Content Signals directive lets you add a line telling crawlers not to use your content for AI, and John Mueller confirmed it does nothing. No crawler or LLM honors it, and it only adds “bloat and future maintenance” to your robots.txt. The companies you are trying to stop never agreed to read it.

Cloudflare admits as much in its own announcement: content signals “express preferences; they are not technical countermeasures against scraping. Some companies might simply ignore them.” A line in robots.txt carries only as much weight as each crawler chooses to give it, and for this directive every crawler so far has given it zero. Cloudflare still added Content-Signal: search=yes, ai-train=no to more than 3.8 million sites automatically, by default. So millions of owners now carry a line they never wrote, telling AI not to train on their content, that no AI obeys.

None of this should surprise you, because robots.txt has always run on the honor system. Well-behaved crawlers read your Disallow lines and stay out. The ones you actually worry about were never held back, because it was never built to hold anyone back. It is a note taped to an unlocked door reading “staff only.” The Content Signals directive just asks for a bigger favor, from companies with every commercial reason to decline, backed by nothing but goodwill.

So separate the two things you have been treating as one. If you want to stop a crawler from reaching something, that is an access decision, and you enforce it with a login, a paywall, or bot management that blocks by verified identity. If you only want a say in what happens to your content after it is fetched, know that today you have a preference and nothing more, and do not build a plan on it. You can’t be maximally findable and also wall your best work off from the systems that decide what gets found. If a crawler can reach it, a model can learn from it.

There is no line you can add to robots.txt that opts you out of AI. There is only the older, harder question of what you are willing to put on the open web, and where, and that one you have to answer yourself.