✳flâneur — a map of the web's best reading

Introducing Cosmos 1.0

astronomer.io · 1,973 words · saved by 1 readers

Apache Airflow, often hailed as the “swiss army knife” of data engineering, is an open-source platform that enables the creation, scheduling, and monitoring of complex data pipelines. A typical pipeline is responsible for extracting, transforming, then loading data - this is where the name “ETL” comes from. Airflow has, and always will, have strong support for each of these data operations through its rich ecosystem of providers and operators. In recent years, dbt Core has emerged as a popular transformation tool in the data engineering and analytics communities. And this is for good reason: it offers users a simple, intuitive SQL interface backed by rich functionality and software engineering best practices. Many data teams have adopted dbt to support their transformation workloads, and dbt’s popularity is fast-growing. The best part: Airflow and dbt are a match made in heaven. dbt equips data analysts with the right tool and capabilities to express their transformations. Data enginee

Introducing Cosmos 1.0 ← Blog | JUL 26 2023 Introducing Cosmos 1.0: the best way to run dbt Core in Airflow Julian LaNeve CTO 8 min read | Share this Article → Note: Since this post was first published in July 2023, thousands of data teams have adopted Cosmos and it is now downloaded over 1.3 million times per month . Additionally, you can now deploy dbt projects to Astro , our managed data platform, significantly reducing the complexity and fragmentation associated with handling multiple data tools. Apache Airflow®, often hailed as the "swiss army knife" of data engineering, is an open-source

Explore this link on the map →

related reading