Releasing Alpaca-30B - by abu - brainwork

A guide on how I fine-tuned Alpaca 30B and how to use it

You may have heard something about a Llama over the past couple weeks. Llama is an open-source (ish) large language model from Facebook. Similar to Stable Diffusion, the open source community has rallied to make Llama better and more accessible. A recent paper from the Tatsu Lab introduced Alpaca, a "instruction-tuned" version of Llama. You can think of Llama as the original GPT-3. To effectively prompt Llama requires that you treat it like autocomplete i.e having the user write the first couple words of their desired output. Instruction-tuning, however, teaches the base model (Llama) to…

related reading

• Stanford CRFMcrfm.stanford.edu
• The Llama Hitchiking Guide to Local LLMs – hackerllamaosanseviero.github.io
• Tutorial: How to Finetune Llama-3 and Use In Ollama | Unsloth Documentationdocs.unsloth.ai
• Fine-Tuning Llama-2: Tailoring Models to Unique Applicationsanyscale.com
• GitHub - rasbt/LLMs-from-scratch: Implement a ChatGPT-like LLM in PyTorch from scratch, step by stepgithub.com
• Accelerating LLaMA with Fabric: A Comprehensive Guide to Training and Fine-Tuning LLaMA - Lightning AIlightning.ai
• 2305.14314arxiv.org
• Tinkerthinkingmachines.ai
• Fine-tuning LLMs Guide | Unsloth Documentationdocs.unsloth.ai
• Fine-tune Llama 2 with DPOhuggingface.co
• Anatomy of a Modern Finetuning APIbenanderson.work
• [2203.02155] Training language models to follow instructions with human feedbackarxiv.org