AI Is Learning at the Wrong Level of Abstraction — Matthieu Wyart - YouTube

Follow along using the transcript.

This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlst Why can deep networks discover abstractions that shallow models miss? Statistical physicist Matthieu Wyart joins Tim Scarfe to argue that the answer lies in the hidden hierarchy of data. Language and images are built from parts within parts; depth lets a network recover those coarse-grained variables and escape the curse of dimensionality. The conversation moves from jamming transitions and rough loss surfaces to Chomsky, context-free grammars and machine creativity. Wyart…

saved by

• Ishaan Panigrahi

related reading

• Strange Geometric Shapes Found Inside AIs — Tom McGrathyoutube.com
• Next-Latent Prediction Transformers Learn World Models feat. Jayden Teoh from MITyoutube.com
• RLMs are Compositional Generalizers with Alex Zhang from MITyoutube.com
• Recursive Language Models — Alex Zhang, MIT PhDyoutube.com
• Why Sam Altman is against a single superintelligenceyoutube.com
• AI researchers debate how close we are to recursive self-improvementyoutube.com
• AI researchers talk continual learning over a game of HORSEyoutube.com
• What if not Circuits? — LessWronglesswrong.com
• Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
• What Is ChatGPT Doing … and Why Does It Work?-Stephen Wolfram Writingswritings.stephenwolfram.com
• Yann LeCun: World Models: Enabling the next AI revolutionyoutube.com
• Deep Dive into LLMs like ChatGPTyoutube.com