Your Website Gets One Chance to Opt Out of AI Training

OpenAI, Google and Microsoft want to honor the same signal websites use to reject the use of their content for AI training. That could make provider-specific instructions less necessary. The companies won’t roll out support all at once.

The proposal uses robots.txt, the file website owners use to give crawlers instructions. Crawlers are programs that automatically visit web pages. According to OpenAI’s announcement, the joint approach should make it easier for publishers to communicate the same preference to multiple AI providers.

The new signal is technically called __usage_ai-training. The proposal submitted to the Internet Engineering Task Force (IETF) explains how a website can use it to specify, section by section, whether its content may be used to train generative AI.

A publisher could, for example, set a site-wide preference against training, then make an exception for a specific section. The proposal builds on robots.txt’s existing structure rather than requiring an entirely new file.

This is still a draft proposal, not a finalized internet standard. Participation is also voluntary. The announced support makes the instruction usable with participating providers, but does not automatically require every AI company to follow it.

A preference about how content may be used is not the same as technically blocking visitors. On its own, the new signal does not lock anyone out of a website. That requires separate measures that actually reject requests.

The distinction also applies to protection against AI crawlers. Bot-management services can identify and block traffic, while robots.txt communicates instructions to parties that process them.

What’s more, the joint signal is specifically about training: changing what a generative AI model stores in its parameters as it learns. Consulting a web page to support a current answer is a different use. Website owners will therefore need to keep checking which settings apply to which use.

OpenAI plans to introduce the signal in the coming months. The company says its existing settings for its own crawlers will remain in effect.

Google is aiming for support in 2027. If a website uses the shared training preference and has not made an explicit choice for Google-Extended, Google plans to treat it as the equivalent setting for Google-Extended. Specific instructions for Google will still be possible. Google also ties permission for Google-Extended to the ability to appear as a source in Gemini Apps answers. So training and source use are not separated in the same way by every provider.

According to The Verge, Anthropic is also joining and already supports the signal. Even so, website owners should not assume that a single new rule will have the same effect immediately across all the providers mentioned.

Microsoft is introducing separate choices for training and grounding. Here, grounding means an AI service uses web content to support a generated answer. Microsoft says websites can make these choices without losing their eligibility for regular Bing search results.

Microsoft will begin processing the new preferences when it recrawls pages from February 1, 2027. March 1, 2027, also matters for websites currently using NoArchive or NoCache: after that date, Microsoft will no longer interpret those legacy settings as preferences for training or grounding. New training preferences must be published before then.

According to Microsoft, sites that want to keep restrictions on grounding must publish new page-level settings and record a temporary preference in Bing Webmaster Tools in time. That temporary arrangement is intended for sites using NoArchive or NoCache on December 15, 2026.

For publishers, the impact of AI traffic matters alongside access. Our earlier explanation of the ratio of AI crawls to visitors sent back explores that trade-off. The new signal sets a usage preference; it does not provide compensation or guarantee visitor traffic.