DuckLake: SQL as a Lakehouse Format – DuckDB

duckdb.org · 3,530 words · saved by 1 readers

TL;DR: DuckLake simplifies lakehouses by using a standard SQL database for all metadata, instead of complex file-based systems, while still storing data in open formats like Parquet. This makes it more reliable, faster, and easier to manage. The first part of the blog post is shared with the DuckLake manifesto. Jump to the DuckLake extension section to read the rest. Innovative data systems like BigQuery and Snowflake have shown that disconnecting storage and compute is a great idea in a time where storage is a virtualized commodity. That way, both storage and compute can scale independently and we don't have to buy expensive database machines just to store tables we will never read. At the same time, market forces have pushed people to insist that data systems use open formats like Parquet to avoid the all-too-common hostage taking of data by a single vendor. In this new world, lots of data systems happily frolic around a pristine “data lake” built on Parquet and S3 and all was well.

Mark Raasveldt and Hannes Mühleisen 2025-05-27 | 20 min TL;DR: DuckLake simplifies lakehouses by using a standard SQL database for all metadata, instead of complex file-based systems, while still storing data in open formats like Parquet. This makes it more reliable, faster, and easier to manage. The first part of the blog post is shared with the DuckLake manifesto. Jump to the DuckLake extension section to read the rest. Background Innovative data systems like BigQuery and Snowflake have shown that disconnecting storage and compute is a great idea in a time where storage is a…

DuckLake: SQL as a Lakehouse Format – DuckDB

related reading