Google Plans One robots.txt Line to Opt Out of AI Training

Google will honor one shared robots.txt signal for AI training in 2027. Here's what the new line does, what it doesn't, and whether to add it.

Have you ever tried to tell every AI company the same thing about your website? You'd need a separate rule for Google, another for OpenAI, another for Microsoft, and a reminder to check them all every time a new bot shows up.

On October 30, 2026, Google said it plans to make that a one-line job. It intends to honor a shared AI training signal in robots.txt, built on work from the IETF, so a single opt-out can reach several AI providers at once.

That sounds like good news. But the details matter, because the new line only covers training, it isn't live yet, and the standard behind it isn't finished.

This post walks through what Google actually announced, what the line looks like, and a simple way to decide whether you want it on your site.

Key Takeaways

• Google announced on October 30, 2026 that it aims to support a shared AI training control in robots.txt in 2027.
• The signal is a policy token, __usage_ai-training, written in the User-Agent line of a normal robots.txt group.
• If you opt out with the token and don't name Google-Extended, Google will treat that as a Google-Extended disallow.
• Google says Google-Extended doesn't affect inclusion in Google Search or act as a ranking signal, so training opt-outs and search visibility stay separate decisions.
• The standard is an individual IETF draft, so nothing in your robots.txt needs to change today.
• Your real decision is whether you want your content used to train AI models, and that is a business call, not a technical one.

What Did Google Announce?

In a Search Central post by John Mueller, Google said it will support a shared control that lets websites express AI training preferences across multiple AI providers in a single step. It is based on work from the AI Preferences (AIPREF) working group at the Internet Engineering Task Force.

In short: Google plans to read one common robots.txt signal for AI training, instead of only its own Google-Extended token. The timeline is early 2027, according to Google's crawler documentation.

Google also says its existing company-specific controls keep working alongside the new one. Nothing you've set up today stops working.

What Does the New Line Look Like?

The proposal comes from an IETF Internet-Draft written by Kevin Kelley of Anthropic. It adds no new directives. It reuses the normal robots.txt rules, and the policy token goes where a bot name would normally go.

Here's the draft's own example. To say "no AI training anywhere except my press kit", you'd write:

• User-Agent: __usage_ai-training
• Disallow: /
• Allow: /press-kit/

A few rules in the draft are worth knowing. A wildcard group (User-Agent: *) doesn't count as a training preference. If no group names the token, you've expressed no preference at all. And the token isn't a crawl directive: it states what you want done with your content, not who may fetch it.

How Does It Work With Google-Extended?

Google-Extended is the token Google already offers. Google's documentation says it lets you manage whether crawled content is used to train future Gemini models, and for grounding in Gemini Apps.

The new rule is simple. If you opt out of training using the shared token and don't explicitly specify Google-Extended, Google will treat it as if you had disallowed Google-Extended. If you name Google-Extended yourself, that explicit rule takes precedence.

That keeps you in control. One line can cover everyone, and a separate rule can still make an exception for Google.

Will Opting Out Hurt Your Google Rankings?

No, according to Google. Its documentation states that Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal. The shared token is meant to behave the same way for training.

There is one trade-off to weigh. Google's post says sites that specifically allow Google-Extended are eligible to appear in responses and receive clicks through grounding in Gemini Apps. Opt out entirely, and you step out of that.

Neither of Google's pages says how the new signal relates to AI Overviews or AI Mode, according to coverage of the announcement. For that, Search Console already has a separate opt-out, which we covered in Google Now Lets You Opt Out of AI Answers. Should You?.

Who Else Is Signing Up?

Google says it aims to support the control along with other AI providers, but it doesn't name them. The wider reporting fills in some gaps.

According to Search Engine Roundtable and Relevant Audience:

• Microsoft Bing announced on the same day that it is adopting the shared preference.
• Anthropic, whose engineer wrote the draft, says it has started implementing support.
• Google tied the idea to the EU Code of Practice for general-purpose AI, which recognizes robots.txt as a way to express preferences.

Notice what's missing: a commitment from every major AI company. Participants in the IETF draft contribute as individuals, and the draft says their affiliations are for identification only.

How to Decide: A Three-Question Check

You don't need to rewrite anything this week. You do need an opinion, so you're ready when the signal goes live. Ask yourself:

1. Is your content a product? If you sell research, courses, premium articles or data, you probably want an opt-out for most or all of it.
2. Do you want AI assistants to know your brand? If you win customers when chatbots recommend you, blocking training may cost you awareness. Allow the pages that explain what you do.
3. Do you have a section that should stay out? Pricing experiments, member areas and client portals are better kept private through a login, not just a robots.txt line.

Most local and service businesses land on a middle path: open marketing pages, protected everything else. That's the same split we describe in our post on blocking AI bots in robots.txt.

The Policy in Action: A Quick Example

Take a software company with a public blog, a help centre and a customer-only knowledge base. Its goals are different for each section.

Once the shared signal is live, the draft's rules would let them write:

• Blog: allow, because the company wants AI tools to learn its positioning.
• Help centre: allow, so assistants give customers accurate answers.
• Knowledge base: disallow for training, and also keep it behind a login.

That is one set of path rules, read the same way by every provider that supports the token. This is the whole point of the proposal.

Where This Breaks Down

Treat the announcement with some caution. Three limits stand out.

• It's a draft. The IETF document is an individual submission, not endorsed by the IETF, and it expires on May 3, 2027 unless it is updated.
• It's contested. Nate Hake, who has followed the IETF process, told Search Engine Roundtable the proposal isn't finished and that significant disputes remain.
• It's voluntary. A preference signal only works with companies that choose to respect it. Private agreements may override it, according to the draft.

If your content is truly sensitive, the safe move is still a login or a paywall. A preference is a request, not a lock.

What to Do This Week

1. Open yourdomain.com/robots.txt and read it top to bottom. Note what's there and who added it.
2. Check that you haven't blocked Google-Extended or other AI bots by accident, which a plugin or an old developer may have done.
3. Write down, in one sentence, what you want AI companies to do with your content.
4. Put a reminder in your calendar for early 2027 and watch Google's common crawlers page for the launch.
5. Don't add the __usage_ai-training line until providers confirm support. An unsupported line does nothing today.

FAQs

What is the __usage_ai-training token?

It's a policy token proposed in an IETF draft, written in the User-Agent line of a robots.txt group. Disallow paths under it mean you prefer no AI training on those pages; Allow paths mean you're fine with it. It expresses a preference, not a crawl rule.

When will Google support the shared AI training control?

Google says it aims to support it in 2027, and its crawler documentation narrows that to early 2027. There's no firm date yet, and the line has no effect on Google until support launches.

Does blocking AI training hurt my SEO?

Not according to Google. It says Google-Extended doesn't affect inclusion in Google Search and isn't a ranking signal. The trade-off is Gemini: only sites that allow Google-Extended are eligible to appear in Gemini Apps responses through grounding.

Do I still need Google-Extended?

Yes, if you want Google-specific rules. Google says existing company-specific controls keep working alongside the shared one, and an explicit Google-Extended rule takes precedence over the shared token.

Conclusion

The idea to remember: Google is moving toward one shared robots.txt signal for AI training, but it's a 2027 plan built on an unfinished draft. It's a preference, not a lock.

This week, read your robots.txt, decide what you want AI companies to do with your content, and keep your search visibility separate from that choice.

If you'd like help working out which pages should be open to AI search and which should not, our GEO optimisation and SEO teams can look at it with you. You can get a free quote any time.

Sources

• Google Search Central: Supporting a shared AI training control for websites
• IETF draft: Training preference signal (draft-kelley-aipref-training-preference-signal)
• Google Search Central: Google's common crawlers
• Search Engine Roundtable: Google AI training control
• Relevant Audience: Google shared AI training robots.txt signal

About the author

David Banjo

Founder & CEO, Binatec Agency

David Banjo is an entrepreneur and web developer, and the Founder & CEO of Binatec Agency, where he helps businesses scale through custom software, digital products, and e-commerce growth strategies. He is also the creator of ProposelyAI, an AI-powered platform designed to streamline and automate proposal generation for freelancers and agencies. Recognized for his work in web development, AI workflow automation, and growth marketing, David equips brands with the modern tools needed to optimize performance and expand their digital reach.

Follow