Tests unable to distinguish AI from human reviews

Share post:

AI-generated restaurant reviews can now pass the Turing test, successfully fooling both human readers and automated detectors, according to a new study conducted by Yale School of Management professor Balázs Kovács. This groundbreaking research, published in the Springer journal *Marketing Letters*, exposes a significant vulnerability in the trustworthiness of online review systems, which are integral to consumer decision-making.

The study involved two experiments with 301 participants who were native English speakers from the United States, Canada, the United Kingdom, Ireland, or Australia. The average age of participants was 47, and 57% were female. In the first experiment, participants were presented with a mixture of real and AI-generated Yelp reviews and were only able to identify the source correctly about 50% of the time—no better than chance. The second experiment, featuring entirely fictional reviews created by GPT-4, found that participants mistakenly believed the AI-generated reviews were written by humans 64% of the time.

Moreover, Kovács tested leading AI detectors, such as Copyleaks, which erroneously classified all AI-generated reviews it was tested on as human-generated. Even GPT-4 failed to reliably distinguish its own AI-generated content from human-written material, with most assessments suggesting little difference between the two.

These findings highlight the challenges and potential risks posed by sophisticated AI in generating credible fake content, posing threats to the integrity of online platforms and the businesses that depend on genuine customer reviews. The study underscores the need for review platforms to enhance their verification processes and for regulators to consider steps to ensure transparency and authenticity in online reviews.


Related articles

ChatGPT mobile mania: Why users are flocking to ChatGPT Plus

On the day OpenAI unveiled GPT-4o, ChatGPT's mobile app saw a staggering 22% spike in revenue, marking its...

Study reveals the “disappearing internet”

In a surprising revelation, new research by the Pew Research Center indicates that the internet may not be...

Scarlett Johansson – did OpenAI use HER voice?

Hollywood star Scarlett Johansson expressed shock and anger after a new OpenAI chatbot debuted with a voice eerily...

Google criticizes Microsoft’s security practices in new report

Google has publicly criticized Microsoft for a series of security missteps, suggesting that organizations might consider more secure...

Become a member

New, Relevant Tech Stories. Our article selection is done by industry professionals. Our writers summarize them to give you the key takeaways