flâneur

Why Hardware-Software Co-Design Is AI's Real 100x: Dylan Patel of SemiAnalysis - YouTube

youtube.com · 11,280 words · saved by 1 readers

Dylan Patel of SemiAnalysis explores the evolution of the semiconductor industry, from early experiences with family businesses to analyzing complex global supply chains. The discussion shifts to the critical role of hardware-software co-design in driving inference efficiency, benchmarking methodologies, and the future of computational infrastructure in an AI-dominated landscape. Follow along using the transcript. Dylan Patel of SemiAnalysis explores the evolution of the semiconductor industry, from early experiences with family businesses to analyzing complex global supply chains. The discussion shifts to the critical role of hardware-software co-design in driving inference efficiency, benchmarking methodologies, and the future of computational infrastructure in an AI-dominated landscape. Follow along using the transcript.

Dylan Patel, founder of SemiAnalysis, argues the biggest gains in AI don't come from faster chips, they come from software-hardware co-design. Optimizing the model, the kernels, and the silicon together turns a 2x here and a 2x there into 100x. He explains why DeepSeek's experts were shaped for Nvidia's Hopper (and why TPUs struggle to run it), why OpenAI's sparser models and Anthropic's denser ones pull them toward different hardware, and why the so-called CUDA moat was never really about CUDA. Dylan breaks down InferenceX, his living benchmark that runs the latest models on over $50M of…

saved by

related reading