AI's new lock: please don't

The real news

The story, straight

Websites have visitors that aren't people. Automated programs called crawlers collect their pages.

AI companies can use collected text to train models: teaching software patterns from examples.

Robots.txt is a plain text file at a website's root, its top-level address. Crawlers read it for instructions about accessing the site.

Think of a sign beside the entrance, not a locked door. A crawler can ignore it.

Now there's a proposed shared signal for that file, letting publishers express AI-training preferences across providers instead of setting them separately for each company. Kevin Kelley, Anthropic's technical lead for standards, proposed it through the Internet Engineering Task Force.

It remains a draft, not a finished standard.

Anthropic has adopted the signal. OpenAI says implementation will come in the coming months.

Google and Microsoft plan support next year. Meta supports the idea but hasn't said it will honour the signal.

Applauding the lock isn't the same as agreeing to use it.

The companies' controls aren't identical. Microsoft describes separate choices for training and grounding, where AI consults sources to help answer accurately; sites would still appear in Bing results.

Google already has separate training controls. OpenAI says the new signal won't replace the robots.txt controls it already honours.

Lil'meme's security upgrade: an enormous paper padlock. Excellent visibility.

Questionable resistance to fingers.

The useful upside is simpler communication: fewer separate settings for site owners. But an easier way to say no isn't an enforced barrier.

Keep that distinction when choosing protections for your site.

Source: The Verge, 30 October 2026.

Like or repost if your security system deserves better than a thank-you note.

Thanks for supporting Lil'meme. Your support is appreciated.