Answer.AI - A few tips for working on high-surface-area problems
answer.ai · 1,204 words · saved by 2 readers
A few tips for working on things that involve a lot of moving pieces, that have a high “surface area”.
A few tips for working on high-surface-area problems – Answer.AI Note A note from Johno I’ve been writing brief notes on my own blog for years. I’ll be posting on the Answer.AI blog too now that I work here – here’s what is hopefully the first of many posts. This one is just a few thoughts inspired by some recent work, which I hope you’ll find interesting. I’d love to hear what you think; you can find me here on Twitter/X . Some of the many questions/directions we faced when trying to get QLoRA working with FSDP - an example of a high-surface-area problem! Some problems are fairly well-defined
saved by
related reading
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- Headroom for AI development – Machine Learning (Theory)hunch.net
- Making Deep Learning go Brrrr From First Principleshorace.io
- Extending Context is Hard | kaiokendevkaiokendev.github.io
- How to Land a Frontier Lab Jobvladfeinberg.com
- Non_Interactive – Software & MLnonint.com
- An Opinionated Guide to ML Researchjoschu.net
- Study Guide — LessWronglesswrong.com
- Tips for Empirical Alignment Research — AI Alignment Forumalignmentforum.org
- Debugging Reinforcement Learning Systemsandyljones.com
- A Recipe for Training Neural Networkskarpathy.github.io
- A Recipe for Training Neural Networkskarpathy.github.io