Prompt Tuning and Prefix Tuning

The original concept of prompt tuning refers to changing the prompt to the LLM to achieve better modeling results. However, it now means adding learnable prompt tokens to pre-trained models, which are inserted either to input embeddings only or to multiple intermediate layers. The following is an example of hard prompt tuning to achieve a better translation result: Hard prompt tuning is where you directly change the discrete input tokens which are not differentiable. Soft prompt tuning concatenates the embeddings of the input tokens with a trainable tensor that can be optimized via backprop to improve the modeling performance on a target task. You can train a small model for a specific task that learns to generate an output that performs the model’s overall performance. After learning a soft prompt, you have to supply it as a prefix when performing the specific task you finetuned the model on. This allows the model to tailor its responses to that particular task. Moreover, we can have

The original concept of prompt tuning refers to changing the prompt to the LLM to achieve better modeling results. However, it now means adding learnable prompt tokens to pre-trained models, which are inserted either to input embeddings only or to multiple intermediate layers. Hard Prompt Tuning § The following is an example of hard prompt tuning to achieve a better translation result: 1) "Translate the English sentence '{english_sentence}' into German: {german_translation)" 2) "English: '{english_sentence}' | German: (german_translation}" 3) "From English to German: '{english_sentence}' -> {g

related reading

• Prefix-Tuning: Optimizing Continuous Prompts for Generationarxiv.org
• Conditioning, Prompts, and Fine-Tuning — LessWronglesswrong.com
• Patterns for Building LLM-based Systems & Productseugeneyan.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• Recent Advances in Language Model Fine-tuningruder.io
• Mastering LLM Techniques: Customization | NVIDIA Technical Blogdeveloper.nvidia.com
• Fine-tuning LLMs Guide | Unsloth Documentationdocs.unsloth.ai
• Model optimization | OpenAI APIplatform.openai.com
• Anatomy of a Modern Finetuning APIbenanderson.work
• LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelsarxiv.org
• Prompt Engineering | Kagglekaggle.com
• Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com