SignalStack
Back to feed

The biggest names in AI are backing a new way to opt out of scraping

Read original

AI summary

Major AI companies are adopting a universal opt-out standard, allowing publishers to block scraping across all platforms with a single setting instead of managing individual requests.

AI-generated summary. Check the original reporting for context and accuracy.

Source excerpt

This is the available feed excerpt, which may be shortened or in the publisher’s original language.

OpenAI, Google, Microsoft, and Anthropic say they will adopt a new standard that publishers can use to opt out of being scraped by AI, as reported earlier by Search Engine Roundtable. With this control, publishers can apply their AI training preferences across multiple AI companies, instead of setting them individually. The shared "policy token" signal is proposed by Kevin Kelley, a technical lead for standards at Anthropic, as part of the Internet Engineering Task Force (IETF). It will live within a website's robots.txt file, which has long signaled which web crawlers can and can't scrape a site - whether for search indexing, archiving, or … Read the full story at The Verge.

About priority

An automated sorting aid based on source trust, keywords, and entities, on a scale of 1–12. High: 9–12; medium: 6–8; low: 1–5. It does not measure urgency, verify reporting, or indicate editorial endorsement.

More in this topic