News organizations block AI crawler bots amid copyright concerns

Share post:

20% of the world’s top 1000 websites are now blocking AI crawler bots, which are used to collect web data for AI services, according to Originality.AI.

The move comes after OpenAI introduced its GPTBot crawler in early August. The GPTBot was designed to collect data from the web to improve future AI models. However, many websites, including the New York Times, Reuters, and CNN, blocked the GPTBot, concerned that it could be used to scrape their content without permission.

The blocking of AI crawler bots is a sign of the growing tension between websites and AI companies over the use of data. While AI companies argue that they need to collect data to train their models, websites are concerned about protecting their content and intellectual property.

According to Originality.AI, the percentage of the top 1000 websites blocking OpenAI’s ChatGPT bot surged from 9.1% on August 22 to 12% on August 29. The situation is further complicated by the lack of clear legal guidelines governing the use of AI crawler bots. As a result, websites are taking matters into their own hands by blocking these bots.

The sources for this piece include an article in Axios.


Related articles

Microsoft’s AI success may spell defeat for it’s climate goals

Microsoft's ambitious strides in AI technology are now posing a significant challenge to its own climate goals, as...

OpenAI’s Chief Scientist Ilya Sutskever Departs Company

Ilya Sutskever, co-founder and chief scientist of OpenAI, has officially announced his departure from the company. This move...

OpenAI snubs Microsoft, launching GPT-4o only on macOS

OpenAI, despite Microsoft's substantial $10 billion investment, has chosen to release its new ChatGPT app exclusively on macOS,...

Apple to integrate ChatGPT into iPhones

Apple Inc. is on the brink of solidifying a deal with OpenAI to integrate the ChatGPT technology into...

Become a member

New, Relevant Tech Stories. Our article selection is done by industry professionals. Our writers summarize them to give you the key takeaways