How to Generate and Use Synthetic Data for Finetuning

eugeneyan.com · 8,536 words · saved by 1 readers

It is increasingly viable to use synthetic data for pretraining, instruction-tuning, and preference-tuning. Synthetic data refers to data generated via a model or simulated environment, instead of naturally occurring on the internet or annotated by humans. Relative to human annotation, it’s faster and cheaper to generate task-specific synthetic data. Furthermore, the quality and diversity of synthetic data often exceeds that of human annotators, leading to improved performance and generalization when models are finetuned on synthetic data. Finally, synthetic data sidesteps privacy and copyright concerns by avoiding reliance on user data or possibly copyrighted content. There are two main approaches to generate synthetic data: Distillation from a stronger model or Self-improvement on the model’s own output. The synthetic data can then be used in pretraining, instruction-tuning, and preference-tuning. Distillation transfers knowledge and reasoning skills from a stronger teacher to a weak

How to Generate and Use Synthetic Data for Finetuning eugeneyan Start Here Writing Speaking Prototyping About How to Generate and Use Synthetic Data for Finetuning [ llm survey ] · 42 min read It is increasingly viable to use synthetic data for pretraining, instruction-tuning, and preference-tuning. Synthetic data refers to data generated via a model or simulated environment, instead of naturally occurring on the internet or annotated by humans. Relative to human annotation, it’s faster and cheaper to generate task-specific synthetic data. Furthermore, the quality and diversity of synthetic da

How to Generate and Use Synthetic Data for Finetuning

related reading