In July 2020, OpenAI launched GPT-3, an artificial intelligence language model that sparked excitement about computers but sometimes proved to be foul-mouthed and toxic.
Before it was licensed to developers, OpenAI published an essay in May 2020 with tests showing that GPT-3 has a generally low opinion of Black people and exhibits sexism and other forms of bias.
Despite these findings, OpenAI announced plans to commercialize the technology – a sharp contrast to the way OpenAI handled an earlier version of the model, GPT-2, in 2019.
Partners from the education sector have published several studies on how large language models can be misused and have a negative impact on society.
In the recently published paper highlighting ways to reduce the toxicity of GPT-3, OpenAI has published tests showing that the base version of GPT-3 calls some people animals and associates white people with terms like “supremacy” and “superiority,” perpetuating long-held stereotypes and dehumanizing non-white people.
Text generated by large language models quickly approaches a text that originates from a human being but still does not understand things that require reason, which almost all people understand.
Several researchers have found that attempts to refine and eliminate the bias of GPT-3 models can ultimately harm marginalized people.
Researchers say the problem stems in part from people misjudging data on the toxicity and non-toxicity of language, leading to prejudice against people who use language differently than white people.
Researchers say this can lead to self-stigmatization and psychological damage, forcing people to change codes.
Researcher Jesse Dodge says the best way to deal with bias and inequality is to improve the data used to train language models, rather than trying to eliminate prejudice in retrospect.
He recommends better documenting the source of training data and recognizing the limitations of texts scraped from the web that might over-represent people who can afford internet access and have time to create a website or comment.
Microsoft researchers surveyed 12 technicians using AI language technology and found that product teams had little idea how the algorithms could go wrong.
The researchers developed an interactive “playbook” that encourages people working on an AI language project to think about and design errors in AI text technology in the early stages. It is being tested within Microsoft to make it a standard tool for product teams.
For more information, read the original story in Arstechnica.