Meta’s LLaMA leaks

Share post:

Meta’s newest large language model family, LLaMA (Large Language Model Meta AI), has surfaced online illegally, along with its weights, and is now freely accessible via torrents.

Since LLaMA was not released as a public chatbot, but rather as an open-source package to which anyone in the AI community can request access, it is intended to be accessible only to approved researchers, government officials, or members of civil society. Unfortunately, anyone can now download it thanks to 4chan, where a member uploaded a torrent file for LLaMA.

On Thursday around 9:45 p.m. IST, user ‘llamanon’ posted on 4chan’s technology board, releasing the LLaMA 7B and 65B models via torrent. The model was made available on 4chan’s AI Chatbot General megathread, which serves as a central location for testing the roleplaying capabilities of the latest AI chatbots.

This torrent link was then added to the LLaMA GitHub page as a pull request with the title’save bandwidth by using a torrent to distribute more efficiently’. This pull request was posted alongside the Google Forms link Meta was using to provide access to the bot, implying a dig at the LLM application process. A second pull request was also submitted to the project, this time with a torrent link to a different set of weights.

According to the leaked version, LLaMa, like other AI models such as OpenAI’s GPT-3, is built on a massive collection of word fragments, or “tokens.” LLaMa can take an input of words and predict the next word to recursively generate more text, just exactly as Meta explains in a February blog post.

LLaMa comes in a variety of sizes, with the LLaMa 65B and LLaMa 33B trained on 1.4 trillion tokens. The model was trained using datasets scraped from Wikipedia, books, academic papers from ArXiv, GitHub, Stack Exchange, and other sites, according to the LLaMA model card.

The sources for this piece include an article in TheRegister.


Related articles

Microsoft’s AI success may spell defeat for it’s climate goals

Microsoft's ambitious strides in AI technology are now posing a significant challenge to its own climate goals, as...

OpenAI’s Chief Scientist Ilya Sutskever Departs Company

Ilya Sutskever, co-founder and chief scientist of OpenAI, has officially announced his departure from the company. This move...

OpenAI snubs Microsoft, launching GPT-4o only on macOS

OpenAI, despite Microsoft's substantial $10 billion investment, has chosen to release its new ChatGPT app exclusively on macOS,...

Apple to integrate ChatGPT into iPhones

Apple Inc. is on the brink of solidifying a deal with OpenAI to integrate the ChatGPT technology into...

Become a member

New, Relevant Tech Stories. Our article selection is done by industry professionals. Our writers summarize them to give you the key takeaways