Meta’s LLaMA leaks

Share post:

Meta’s newest large language model family, LLaMA (Large Language Model Meta AI), has surfaced online illegally, along with its weights, and is now freely accessible via torrents.

Since LLaMA was not released as a public chatbot, but rather as an open-source package to which anyone in the AI community can request access, it is intended to be accessible only to approved researchers, government officials, or members of civil society. Unfortunately, anyone can now download it thanks to 4chan, where a member uploaded a torrent file for LLaMA.

On Thursday around 9:45 p.m. IST, user ‘llamanon’ posted on 4chan’s technology board, releasing the LLaMA 7B and 65B models via torrent. The model was made available on 4chan’s AI Chatbot General megathread, which serves as a central location for testing the roleplaying capabilities of the latest AI chatbots.

This torrent link was then added to the LLaMA GitHub page as a pull request with the title’save bandwidth by using a torrent to distribute more efficiently’. This pull request was posted alongside the Google Forms link Meta was using to provide access to the bot, implying a dig at the LLM application process. A second pull request was also submitted to the project, this time with a torrent link to a different set of weights.

According to the leaked version, LLaMa, like other AI models such as OpenAI’s GPT-3, is built on a massive collection of word fragments, or “tokens.” LLaMa can take an input of words and predict the next word to recursively generate more text, just exactly as Meta explains in a February blog post.

LLaMa comes in a variety of sizes, with the LLaMa 65B and LLaMa 33B trained on 1.4 trillion tokens. The model was trained using datasets scraped from Wikipedia, books, academic papers from ArXiv, GitHub, Stack Exchange, and other sites, according to the LLaMA model card.

The sources for this piece include an article in TheRegister.

Featured Tech Jobs



Related articles

Stack Overflow moderators strike over limited authority to remove AI-generated content

Stack Overflow’s moderators are on strike over a new policy that limits their ability to remove AI-generated content....

Generative AI holds promise for marketers, finds Salesforce study

Over 1,000 marketing professionals from all sectors and sizes are exploring the practical uses of generative AI while...

Altman tips Israel for big role in AI risk mitigation

During a discussion with Israeli President Isaac Herzog, OpenAI CEO Sam Altman voiced his opinion that Israel can... records two million downloads in first week

A new AI platform founded by ex-Google employees Noam Shazeer and Daniel De Freitas with over 10 million...

Become a member

New, Relevant Tech Stories. Our article selection is done by industry professionals. Our writers summarize them to give you the key takeaways