Is AI hitting a wall? - by Rohit Krishnan
substack.com · 2,974 words · saved by 1 readers
“Any fool can know. The point is to understand.” ― Albert Einstein
I'll start at the end. No. It's not. Of course, I can’t leave it at that. The reason the question comes up is that there have been a lot of statements that they are stalling a bit. Even Ilya has said that it is. Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that use s a vast amount of unlabeled data to understand language patterns and structures - have plateaued. Also, as he said at Neurips yesterday: Of course, he’s a competitor now to OpenAI, so maybe it…
related reading
- Will scaling work?dwarkeshpatel.com
- Alignment is not solved but it increasingly looks solvablealigned.substack.com
- There Are No New Ideas in AI… Only New Datasetsblog.jxmo.io
- My picture of the present in AI — LessWronglesswrong.com
- Is AI progress slowing down?normaltech.ai
- Does AI Progress Have a Speed Limit?—Asteriskasteriskmag.com
- Training Imperativesdan.io
- AI progress is about to speed up | Epoch AIepoch.ai
- Scaling: The State of Play in AIoneusefulthing.org
- The least understood driver of AI progress | Epoch AIepoch.ai
- Deep Learning Is Hitting a Wallnautil.us
- Unresolved debates about the future of AIhelentoner.substack.com