Tests unable to distinguish AI from human reviews

Share post:

AI-generated restaurant reviews can now pass the Turing test, successfully fooling both human readers and automated detectors, according to a new study conducted by Yale School of Management professor Balázs Kovács. This groundbreaking research, published in the Springer journal *Marketing Letters*, exposes a significant vulnerability in the trustworthiness of online review systems, which are integral to consumer decision-making.

The study involved two experiments with 301 participants who were native English speakers from the United States, Canada, the United Kingdom, Ireland, or Australia. The average age of participants was 47, and 57% were female. In the first experiment, participants were presented with a mixture of real and AI-generated Yelp reviews and were only able to identify the source correctly about 50% of the time—no better than chance. The second experiment, featuring entirely fictional reviews created by GPT-4, found that participants mistakenly believed the AI-generated reviews were written by humans 64% of the time.

Moreover, Kovács tested leading AI detectors, such as Copyleaks, which erroneously classified all AI-generated reviews it was tested on as human-generated. Even GPT-4 failed to reliably distinguish its own AI-generated content from human-written material, with most assessments suggesting little difference between the two.

These findings highlight the challenges and potential risks posed by sophisticated AI in generating credible fake content, posing threats to the integrity of online platforms and the businesses that depend on genuine customer reviews. The study underscores the need for review platforms to enhance their verification processes and for regulators to consider steps to ensure transparency and authenticity in online reviews.

SUBSCRIBE NOW

Related articles

Startup Claims To Tackle AI Hallucinations In Real Time

San Francisco-based startup Patronus AI has introduced a new platform designed to prevent AI-generated inaccuracies, or “hallucinations,” in...

Canadian Cyber Defence Agency Lists India as a Cybersecurity Threat for First Time

Canada's federal cyber defence agency, the Communications Security Establishment (CSE), has identified India as a cybersecurity threat to...

AI – Where Are We Two Years After ChatGPT Launch? Hashtag Trending Weekend Edition for November 2, 2024

Project Synapse: Exploring the Evolution of AI with Marcel Gagné and Jim Love In this episode, Jim Love introduces...

Cyber Security Today Weekend – Show notes from the panel show

Cybersecurity Today: Monthly Panel Discussion In this month's weekend edition of Cybersecurity Today, host Jim Love assembles a top...

Become a member

New, Relevant Tech Stories. Our article selection is done by industry professionals. Our writers summarize them to give you the key takeaways