🔍 Read the full analysis: What’s Missing From The AI Training Opt-Out Backed By Four Companies? on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
OpenAI, Google, Microsoft and Anthropic have said they will support a shared robots.txt signal asking that a website’s content not be used to train generative AI models. The signal is voluntary, does not cover every use of web content, and is still an Internet-Draft rather than an adopted internet standard; Meta voiced support without committing to honor it.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will honor a shared website instruction asking them not to use a publisher’s content to train generative AI models. The proposed robots.txt signal could replace multiple company-specific training rules, but it is a voluntary request, and each company has set a different timeline for supporting it.
The shared rule appears in an Internet-Draft titled Machine-Readable Training Preference Signal, published by Anthropic standards lead Kevin Kelley. A publisher would place a user-agent group for __usage_ai-training in its robots.txt file and disallow access to the site’s paths for that purpose. The announcement describes a common way to express a preference; it does not make the preference technically enforceable.
Anthropic says it has started implementing support, while OpenAI aims to implement the signal “in the coming months.” Google says it aims to support it in 2027; its crawler documentation specifies “early 2027.” Microsoft says it will honor the training preference from February 1, 2027, as it crawls websites. These are company statements and planned dates, not evidence that all systems already recognize the rule.
Microsoft is also introducing a separate page-level preference for content used to inform AI answers, which it calls AI Grounding. It says that preference and the training signal will take effect on February 1, 2027. Microsoft says its existing NoArchive and NoCache tags will stop counting as training or grounding preferences after March 1, 2027. Those tags and other company-specific crawler controls remain relevant during the rollout, according to the announcements.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
A common instruction could reduce the work publishers face when expressing a preference to multiple AI companies. Under the current approach described in the source material, website operators may need separate rules for OpenAI’s GPTBot, Google-Extended and Anthropic’s ClaudeBot, as well as Microsoft-specific page tags. A shared signal gives participating companies one instruction to interpret, rather than requiring publishers to manage each company’s controls alone.
But the distinction between requesting a restriction and blocking access matters. The draft does not itself prevent a company from crawling a site or create a technical lock. It also concerns training, not every use of published material: it does not cover live web content used to generate AI answers unless a separate preference applies. Microsoft’s grounding control is one example of that separate decision. A training opt-out also does not remove material from models already trained on it or establish payment for publishers.
The arrangements affect different interests for publishers, site operators and readers. A publisher may want to allow material to appear in search while refusing its use for model training, or may make a different choice about AI-generated answers. The separate controls and company-specific timelines mean operators will need to check what each preference actually governs rather than treating the shared signal as a universal “no AI” setting.
Products Worth Considering
Canon SELPHY CP1500 Wireless Compact Photo Printer (Black)
FLEXISPOT EN1-PRO Electric Standing Desk, 79 x 32, Black Walnut
Dell XPS 13 9350 Laptop, 13.4 FHD+ Display, Intel Ultra 7 256V, 16GB DDR5, 1TB SSD
As an affiliate, we earn on qualifying purchases.
How the Proposal Reached Companies
The signal is based on an individual IETF Internet-Draft associated with the Internet Engineering Task Force’s AI Preferences working group. The draft was published on October 30, and all four companies’ announcements point to it. An Internet-Draft is a proposal under discussion, not an adopted technical standard. The source article reports that the draft has no formal standing in the IETF standards process.
The working group’s status also remains unsettled. At a meeting on October 21, one of its chairs said, “We haven’t declared consensus on anything yet.” That does not prevent companies from choosing to implement a draft, but it means the shared rule should not be described as an approved internet standard. Existing company controls remain in place as firms work toward their stated plans.
Meta’s communications chief, Andy Stone, said the company supports publishers’ ability to express preferences and called a common signal an important step. The statement, as reported in the source material, did not say whether Meta will honor the rule. The material also identified no statements of support from Apple, Amazon, xAI or Perplexity. Their positions should not be inferred from the four companies’ announcements or from names acknowledged in the draft.
““has already started implementing support for this draft””
— Anthropic
Adoption and Enforcement Remain Open
Several details remain unresolved. The source material gives different implementation targets and does not establish when every service operated by each company will recognize the signal. Anthropic says rollout across its services will take time; OpenAI’s “coming months” target is not a calendar date; and Google and Microsoft have later stated schedules. The companies’ announcements do not establish how consistently the preference will be applied across all products or content sources.
The signal is voluntary, and the draft is not an adopted IETF standard. The available material also does not establish whether Meta will implement it or whether other large AI companies will join. Nor does the training rule answer what happens to content already used in training, or settle how publishers can address other uses such as AI answers. Those questions depend on separate company policies and controls.
Watch Rollout Dates and IETF Review
Publishers can review their robots.txt rules and decide separately whether to express preferences about training, AI answers and search. The source recommends keeping existing company-specific rules while providers move to the shared signal, rather than assuming that all crawlers have switched. Microsoft users of NoArchive or NoCache should also track the company’s stated March 1, 2027 changeover for those tags as training or grounding preferences.
The next indicators are whether companies deliver on their announced schedules, publish service-specific implementation details, and clarify how the signal interacts with existing crawler controls. The IETF working group’s further review will show whether the draft advances toward consensus or changes. Until then, the shared rule is a useful common request for participating companies, not a guarantee that content will be excluded from every AI use.
Key Questions
What does the shared AI training opt-out do?
It gives website operators a common robots.txt instruction asking participating companies not to use the site’s content to train generative AI models. It is a preference, not a technical barrier that guarantees a crawler cannot access the site.
Which companies say they will honor it?
OpenAI, Google, Microsoft and Anthropic have announced support, with different implementation schedules. Meta voiced support for publishers expressing preferences but, according to the source material, did not commit to honoring the signal.
Does the rule block AI answers or remove content from existing models?
No. The shared signal concerns training; it does not by itself cover content used for live AI answers or remove material from models already trained. Microsoft has announced a separate AI Grounding preference for content used to inform its AI answers.
Is the signal an official internet standard?
No. It is an IETF Internet-Draft, and the source material says it has no formal standing in the standards process. A working-group chair said consensus had not been declared at an October 21 meeting.
Should publishers remove their existing crawler rules?
The source material advises keeping company-specific controls while providers implement the shared signal. Their current rules may continue to apply during the rollout, and operators should check each provider’s documentation and dates before changing their settings.
Source: ThorstenMeyerAI.com
