GPT-4 breaks AI-guardian defense with natural language prompts

Share post:

Nicholas Carlini, a Google scientist, has demonstrated how OpenAI’s GPT-4 big language model may be used to circumvent AI-Guardian, a safeguard against adversarial attacks on machine learning models.

Carlini utilized GPT-4 to develop code capable of identifying the mask used by AI-Guardian to detect adversarial samples. This enabled Carlini to create hostile cases that could go around the defense.

By directing GPT-4 to create an attack method and explain its workings, Carlini revealed how the chatbot could compromise AI-Guardian’s detection capabilities. Specifically, GPT-4 produced Python code to manipulate images without triggering AI-Guardian’s suspicions. This ability to fool classifiers significantly reduced AI-Guardian’s robustness from 98 percent to a mere 8 percent.

The study reveals machine learning algorithms, such as image recognition systems, are vulnerable to adversarial examples—input that misleads the model’s identification process. Carlini’s revelation of the mask used to identify adversarial samples contradicted AI-Guardian’s technique of establishing a backdoor to reject hostile input, allowing the design of effective adversarial assaults.

“This work shows that GPT-4 can be used as a powerful tool for attacking machine learning models,” said Carlini. “It also raises concerns about the security of AI-Guardian and other similar defenses.”

The sources for this piece include an article in TheRegister.

Featured Tech Jobs


Related articles

Google Bard chats indexed in Google search

Users of Google Bard are reporting that their conversations with Bard are being indexed and appearing in Google...

Jony Ive and OpenAI working on AI hardware partnership

Jony Ive, the former design lead at Apple, is reportedly in talks with OpenAI CEO Sam Altman about...

AI models show promise in hurricane forecasting

New AI-based weather models are showing promise in hurricane forecasting. These AI weather models work by learning patterns...

Spotify launches AI voice translator for podcasts

Spotify is set to take podcasting to a new level with its new AI voice translation feature which...

Become a member

New, Relevant Tech Stories. Our article selection is done by industry professionals. Our writers summarize them to give you the key takeaways