Why didn't we get GPT-2 in 2005?
dynomight.net · 1,681 words · saved by 5 readers
We probably could have
dynomight · Mar 2023 · #science #economics #AI The ancient Romans were never great at building ships and never tried to explore the Atlantic. The basic reason seems to be—why bother? The open ocean has no resources and is a vast plane of death. But imagine that in 146 BC after the Romans killed everyone in Carthage, they found a chest in the basement somewhere. The chest was full of gold and labeled “ Gold from Gold Island which is in the middle of the Atlantic somewhere and has lots of gold ”. It’s plausible the Romans would have figured out how to build great ships and gone to find that isla
saved by
related reading
- Will scaling work?dwarkeshpatel.com
- I. From GPT-4 to AGI: Counting the OOMs - SITUATIONAL AWARENESSsituational-awareness.ai
- AI progress is about to speed up | Epoch AIepoch.ai
- Scaling: The State of Play in AIoneusefulthing.org
- Noam Brown on X: "Implications of Large-Scale Test-Time Compute" / Xx.com
- The Human Skill That Eludes AI - The Atlantictheatlantic.com
- Training Imperativesdan.io
- Society's Technical Debt and Software's Gutenberg Momentskventures.substack.com
- Why didn't DeepMind build GPT3?rootnodes.substack.com
- Let's reproduce GPT-2 (124M)youtube.com
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- How fast is AI improving? - AI Digesttheaidigest.org