China Launches A More Advanced GPT-3 Model

Share post:

After a year, researchers at the Beijing Academy of Artificial Intelligence announced Tuesday the launch of their own generative deep-learning model, Wu Dao, a major breakthrough in AI that can do everything GPT-3 can and more.

For starters, it is enormous: it has been trained on 1.75 trillion parameters, ten times larger than the 175 billion GPT-3 and 150 billion parameters larger than Google’s Switch transformers.

Wu Dao came only three months after the release of version 1.0 in March when the BAAI researchers first developed an open source learning system called FastMoE, which resembles Google’s mix of experts.

This system, which can be operated on PyTorch, enabled the model to be trained on both supercomputer clusters and conventional GPUs.

This allowed FastMoE more flexibility than Google’s system, as the former does not require proprietary hardware such as Google’s TPUs and can therefore run on off-the-shelf hardware supercomputing clusters regardless.

This opens up a lot of possibilities because Wu Dao is multimodal, much like Facebook’s AI against hate speech or Google’s recently released MUM.

BAAI researchers demonstrated Wu Dao’s abilities to perform natural speech processing, text generation, image recognition and image generation during the lab’s annual conference last Tuesday.

The new model can not only write essays, poems and couplets in traditional Chinese but can also generate alt-text based on a static image and near-photorealistic images based on natural language descriptions.

Wu Dao also demonstrated its ability to power virtual idols and predict 3D structures of proteins like AlphaFold.

For more information, read the original story in Engadget.

Featured Tech Jobs


Related articles

How to defend your organization against deepfake content

Advice from the FBI, NSA and CISA on spotting and preventing your firm from being victimized by deepfake audio

Musk stirs controversy with move by X to collect biometrics

Initally biometric data will be collected from X Premium users starting Sept

Identity-based attacks increasing, warns CrowdStrike

Successful identity-based attacks continue to plague IT departments, according to CrowdStrike’s sixth annual Threat Hunting report. Based on an analysis of what they call interactive intrusions — where a threat actor was operating with hands-on-keyboard in a victim’s IT environment for the 12-month period ending June 30 — researchers found: — there was a 62

Many organizations want to limit employee access to AI, survey shows

A large number of organizations are considering or implementing bans on employee use of ChatGPT and other generative AI applications over security, privacy and brand damage concerns, according to a survey by BlackBerry. Three-quarters of 2,000 IT decision-makers in the U.S., Canada, the U.K., France, Germany, the Netherlands, Japan, and Australia said that’s the way

Become a member

New, Relevant Tech Stories. Our article selection is done by industry professionals. Our writers summarize them to give you the key takeaways