Evaluating the role of pretraining dataset size and diversity on single-cell foundation model performance | Nature Methods
The performance of single-cell foundation models is dependent on many factors. This study assesses the effect of the pretraining dataset’s size and diversity, revealing potential challenges in pursuing consistent improvement by naively scaling up pretraining data.
Lopez, R., Regier, J., Cole, M. B., Jordan, M. I. & Yosef, N. Deep generative modeling for single-cell transcriptomics. Nat. Methods 15, 1053–1058 (2018). Article CAS PubMed PubMed Central Google Scholar Demetci, P., Santorella, R., Sandstede, B., Noble, W. S. & Singh, R. Scot: single-cell multi-omics alignment with optimal transport. J. Comput. Biol. 29, 3–18 (2022). Article CAS PubMed PubMed Central Google Scholar van Dijk, D. et al. Recovering gene interactions from single-cell data using data diffusion. Cell 174, 716–729 (2018). Google Scholar Lotfollahi, M., Wolf, F. A. & Theis,…
saved by
related reading
- Accurate predictions on small data with a tabular foundation modelnature.com
- Advancing regulatory variant effect prediction with AlphaGenomenature.com
- Quantifying the effect of experimental perturbations at single-cell resolutionnature.com
- Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
- scGen predicts single-cell perturbation responsesnature.com
- PREDICTING SINGLE-CELL PERTURBATION RESPONSES FOR UNSEEN DRUGSarxiv.org
- scGPT: toward building a foundation model for single-cell multi-omics using generative AI | Nature Methodsnature.com
- Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselinesnature.com
- Towards the explainability of protein language modelsnature.com
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
- [2605.12715] Scaling Laws for Mixture Pretraining Under Data Constraintsarxiv.org
- On the Opportunities and Risks of Foundation Modelsarxiv.org