A Call to Build Models Like We Build Open-Source Software
1 Introduction 2 A brief explanation of open-source software development 3 Community-developed and continually-improved models 3.1 Incremental and cheaply-communicable updates 3.2 Merging models 3.3 Vetting community contributions 3.4 Versioning and backward compatibility 3.5 Modularity and distribution 4 An example future 5 Conclusion
A Call to Build Models Like We Build Open-Source Software A Call to Build Models Like We Build Open-Source Software Colin Raffel December 8th, 2021 colinraffel.com/blog Contents 1 Introduction 2 A brief explanation of open-source software development 3 Community-developed and continually-improved models 3.1 Incremental and cheaply-communicable updates 3.2 Merging models 3.3 Vetting community contributions 3.4 Versioning and backward compatibility 3.5 Modularity and distribution 4 An example future 5 Conclusion Introduction Over the past decade or so, it has become increasingly common to use tr
saved by
related reading
- Keep the Tokens Flowing: Lessons from 16 Open-Source RL Librarieshuggingface.co
- A Recipe for Training Neural Networkskarpathy.github.io
- Recent Advances in Language Model Fine-tuningruder.io
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- Introducing Marin: An Open Lab for Building Foundation Models | Marinmarin.community
- Open Weights isn't Open Trainingworkshoplabs.ai
- A Recipe for Training Neural Networkskarpathy.github.io
- Malleable software in the age of LLMsgeoffreylitt.com
- What I learned from looking at 900 most popular open source AI toolshuyenchip.com
- Let's build GPT: from scratch, in code, spelled out.youtube.com
- GitHub - rasbt/LLMs-from-scratch: Implement a ChatGPT-like LLM in PyTorch from scratch, step by stepgithub.com
- Training Imperativesdan.io