[2403.02327] Model Lakes

Abstract:Given a set of deep learning models, it can be hard to find models appropriate to a task, understand the models, and characterize how models are different one from another. Currently, practitioners rely on manually-written documentation to understand and choose models. However, not all models have complete and reliable documentation. As the number of models increases, the challenges of finding, differentiating, and understanding models become increasingly crucial. Inspired from research on data lakes, we introduce the concept of model lakes. We formalize key model lake tasks, including model attribution, versioning, search, and benchmarking, and discuss fundamental research challenges in the management of large models. We also explore what data management techniques can be brought to bear on the study of large model management.

Model Lakes Koyena Pal David Bau Renée J. Miller Northeastern University Northeastern University Northeastern & U. Waterloo Boston, Massachusetts, USA Boston, Massachusetts, USA Waterloo, ON, Canada…

saved by

• Rikard Saqe

related reading

• Retrieve, Merge, Predict: Augmenting Tables with Data Lakeshal.science
• [2011.10427] Dataset Discovery in Data Lakesarxiv.org
• NL.pdfabehrouz.github.io
• 2403.09611.pdfarxiv.org
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• [2511.08579] Training Language Models to Explain Their Own Computationsarxiv.org
• 2409.02908arxiv.org
• Self-Adapting Language Modelsarxiv.org
• Large Language Diffusion Modelsarxiv.org
• 2108.05053arxiv.org
• [2004.05107] Levels of Analysis for Machine Learningar5iv.labs.arxiv.org
• Position: It's Time to Optimize for Self-Consistencytime-for-consistency.github.io