Stanford researchers fine-tuned a seven-billion-parameter variant of Meta’s recently announced LLaMA model using 52,000 instruction-following demonstrations generated by OpenAI’s GPT-3.5 (text-davinci-003).
The Stanford group began with the Meta LLaMA 7B language model, which is the cheapest and smallest of the various LLaMA models available as open-source. Despite having some pre-existing capacity from being trained on a trillion tokens, this language model would fall far behind ChatGPT in most tasks. The primary value and competitive advantage of the GPT models is primarily due to the extensive time and human resources invested in post-training by OpenAI.
According to the researchers, the Stanford team used the AI-generated instructions to train Alpaca 7B, a language model that exhibits many GPT-3.5-like behaviours. In a blind test with input from the Self-Instruct Evaluation Set, both models performed comparably.
When the LLaMA 7B model became operational, the Stanford team asked GPT to take 175 human-written instruction/output pairs and generate 20 more in the same style and format, 20 at a time. This was done automatically using one of OpenAI’s APIs. The team then had 52,000 sample conversations to use for post-training the LLaMA model. After that, the data was used to fine-tune the LLaMA model, which took about three hours on eight A100 cloud processing computers with 80 GB of storage. This was under $100 USD.
Alpaca suffers from the same issues as other language models as a result of the similarities and training, such as hallucinations, toxicity, and stereotyping. Hallucinations, in particular, are more common in the OpenAI model.
The sources for this piece include an article in The-decoder.