ChatGPT used to fix software bugs

Share post:

According to researchers from Johannes Gutenberg University Mainz and University College London, OpenAI’s ChatGPT chatbot is very good at fixing software bugs.

“We find that ChatGPT’s bug fixing performance is competitive to the common deep learning approaches CoCoNut and Codex and notably better than the results reported for the standard program repair approaches,” the researchers write in a new arXiv paper, first spotted by New Scientist.

The researchers used the QuixBugs bug fixing benchmark to evaluate ChatGPT’s performance. Because they were developed prior to 2018, automated program repair (APR) systems appear to be at a disadvantage. ChatGPT was tested against QuixBugs 40 Python-only problems, and the researchers manually checked whether the suggested solution was correct or not.

ChatGPT fixed 19 of the 40 Python bugs, matching CoCoNut (19) and Codex (19). (21). However, standard APR methods only addressed seven of the issues. The researchers discovered that ChatGPT had a success rate of 77.5% with follow-up interactions.

OpenAI also highlights ChatGPT’s dialogue capability in debugging code examples, where it can ask for clarifications and receive hints from a person to arrive at a better answer. It used Reinforcement Learning from Human Feedback to train the large language models that power ChatGPT (GPT-3 and GPT 3.5). (RLHF).

The sources for this piece include an article in ZDNet.

Featured Tech Jobs

SUBSCRIBE NOW

Related articles

AI presents an “extinction level threat” – US Gov’t Report: Hashtag Trending for Tuesday, March 12, 2024

A new US government report warns that AI presents an “extinction level threat to the human species. Elon Musk is outsourcing his Grok AI code. Hackers have breached the Cybersecurity and Infrastructure Security Agency in the US and a researcher shows how to steal a Tesla by leveraging a feature of the Tesla charging stations.

Robot startup uses ChatGPT to enhance its communications and reasoning skills

Humanoid robot startup Figure has secured a significant $675 million investment from a group of high-profile investors, including...

Lawsuit requires Pegasus spyware to provide code used to spy on WhatsApp users

NSO Group, the developer behind the sophisticated Pegasus spyware, has been ordered by a US court to provide...

OpenAI claims New York Times manipulated ChatGPT “fabricate data”

OpenAI has challenged the New York Times' copyright lawsuit, asserting the newspaper manipulated ChatGPT to fabricate evidence. The...

Become a member

New, Relevant Tech Stories. Our article selection is done by industry professionals. Our writers summarize them to give you the key takeaways