AI-generated restaurant reviews can now pass the Turing test, successfully fooling both human readers and automated detectors, according to a new study conducted by Yale School of Management professor Balázs Kovács. This groundbreaking research, published in the Springer journal *Marketing Letters*, exposes a significant vulnerability in the trustworthiness of online review systems, which are integral to consumer decision-making.
The study involved two experiments with 301 participants who were native English speakers from the United States, Canada, the United Kingdom, Ireland, or Australia. The average age of participants was 47, and 57% were female. In the first experiment, participants were presented with a mixture of real and AI-generated Yelp reviews and were only able to identify the source correctly about 50% of the time—no better than chance. The second experiment, featuring entirely fictional reviews created by GPT-4, found that participants mistakenly believed the AI-generated reviews were written by humans 64% of the time.
Moreover, Kovács tested leading AI detectors, such as Copyleaks, which erroneously classified all AI-generated reviews it was tested on as human-generated. Even GPT-4 failed to reliably distinguish its own AI-generated content from human-written material, with most assessments suggesting little difference between the two.
These findings highlight the challenges and potential risks posed by sophisticated AI in generating credible fake content, posing threats to the integrity of online platforms and the businesses that depend on genuine customer reviews. The study underscores the need for review platforms to enhance their verification processes and for regulators to consider steps to ensure transparency and authenticity in online reviews.
