flâneur

100M Token Context Windows — Magic

magic.dev · 1,801 words · saved by 1 readers

Research update on ultra-long context models, our partnership with Google Cloud, and new funding.

100M Token Context Windows Research update on ultra-long context models, our partnership with Google Cloud, and new funding. Magic Team , on August 29, 2024 There are currently two ways for AI models to learn things: training, and in-context during inference. Until now, training has dominated, because contexts are relatively short. But ultra-long context could change that. Instead of relying on fuzzy memorization, our LTM (Long-Term Memory) models are trained to reason on up to 100M tokens of context given to them during inference. While the commercial applications of these ultra-long context

related reading