Engineers of Scale | Sudip Chakrabarti | Substack
In our Engineers of Scale podcast, we relive and celebrate the pivotal projects in Infrastructure Software that have changed the course of the industry. We interview the engineering “heroes” who had led those projects to tell us the insider’s story. For each such project, we go back in time and do an in-depth analysis of the project - historical context, technical breakthroughs, team, successes and learnings - to help the next generation of engineers learn from those transformational projects. We kicked off our first “season” with the topic of Data Engineering, covering the projects that defined and shaped the data infrastructure industry. In our previous episode, we hosted Doug Cutting and Mike Cafarella for a fascinating discussion on Hadoop. In this episode, we are incredibly fortunate to have Reynold Xin, co-creator of Apache Spark and co-founder of Databricks, share with us the fascinating origin story of Spark, why Spark gained unprecedented adoption in a very short time, the tec
Engineers of Scale From Spark to Databricks: Spark's Origins, Innovations, and What's Next - with Reynold Xin 14 2 1× 0:00 Current time: 0:00 / Total time: -51:50 -51:50 Audio playback is not supported on your browser. Please upgrade. From Spark to Databricks: Spark's Origins, Innovations, and What's Next - with Reynold Xin The insider's story of how Spark got created, what made Spark and Databricks successful, and the future of Spark - from Reynold Xin, co-creator of Spark and co-founder of Databricks. Sudip Chakrabarti Dec 13, 2023 14 2 Share Transcript In our Engineers of Scale podcast, we
Explore this link on the map →related reading
- From Spark to Databricks: Spark's Origins, Innovations, and What's Next - with Reynold Xinsudipchakrabarti.substack.com
- What is Apache Spark? | Google Cloudcloud.google.com
- The Friendship That Made Google Huge | The New Yorkernewyorker.com
- Paper Notes: Spark – Cluster Computing with Working Sets – Distributed Computing Musingsdistributed-computing-musings.com
- NYSRGnotes.ekzhang.com
- RDD vs Dataframe vs Datasetlinkedin.com
- Choose Boring Technologyboringtechnology.club
- MapReduce and Spark - Database Systemscs186berkeley.net
- Building and operating a pretty big storage system called S3 | All Things Distributedallthingsdistributed.com
- Spark Architecture: A Deep Dive. Apache Spark is an open-source… | by Amit Joshi | Mediummedium.com
- How the Community Turned Into a SaaS Commercialluminousmen.com
- Infrastructure in '23 | Kleiner Perkinskleinerperkins.com