Parameter-Efficient Fine-Tuning using 🤗 PEFT

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Motivation Large Language Models (LLMs) based on the transformer architecture, like GPT, T5, and BERT have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. They have also started foraying into other domains, such as Computer Vision (CV) (VIT, Stable Diffusion, LayoutLM) and Audio (Whisper, XLS-R). The conventional paradigm is large-scale pretraining on generic web-scale data, followed by fine-tuning to downstream tasks. Fine-tuning these pretrained LLMs on downstream datasets results in huge performance gains when compared to using the pretrained LLMs…

related reading

• LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
• PEFT Method Overview [implementing Adapters in PyTorch] | Eva Korolevaxmarva.github.io
• Finetune LLMs on your own consumer hardware using tools from PyTorch and Hugging Face ecosystem – PyTorchpytorch.org
• Recent Advances in Language Model Fine-tuningruder.io
• Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
• In-depth guide to fine-tuning LLMs with LoRA and QLoRA | Mercity Researchmercity.ai
• [2106.09685] LoRA: Low-Rank Adaptation of Large Language Modelsarxiv.org
• Efficient LLM Finetuning with Unsloth | Modal Docsmodal.com
• Optimizing inference · Hugging Facehuggingface.co
• Fine-tune Llama 2 with DPOhuggingface.co
• Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
• GitHub - NVIDIA-NeMo/Automodel: 🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face supportgithub.com