Google, Anthropic and others to support new AI crawling opt-out standard

Google LLC, Microsoft Corp., Anthropic PBC and OpenAI Group PBC today announced plans to support an upcoming website crawling standard. The technology will enable website operators to block artificial intelligence developers from using their content to train their models. Many websites contain a text file titled “robots.txt” that informs crawlers what content they may access. […] The post Google, Anthropic and others to support new AI crawling opt-out standard appeared first on SiliconANGLE .