Format | Apache Arrow
arrow.apache.org · 375 words · saved by 1 readers
Arrow Format
Apache Arrow Overview Apache Arrow is a multi-language toolbox for building high performance applications that process and transport large data sets. It is designed to both improve the performance of analytical algorithms and the efficiency of moving data from one system or programming language to another. A critical component of Apache Arrow is its in-memory columnar format, a standardized, language-agnostic specification for representing structured, table-like datasets in-memory. This data format has a rich data type system (included nested and user-defined data types) designed to…
saved by
related reading
- Lance Format and LanceDB: Columnar Storage for the Embedding Age | Andrea Bozzo | Blogandreabozzo.github.io
- abseil / Performance Hintsabseil.io
- What is Apache Spark? | Google Cloudcloud.google.com
- Against SQLscattered-thoughts.net
- Spark Architecture: A Deep Dive. Apache Spark is an open-source… | by Amit Joshi | Mediummedium.com
- Pruning for Icebergsnowflake.com
- SPARQLen.wikipedia.org
- RDD vs Dataframe vs Datasetlinkedin.com
- DuckLake: SQL as a Lakehouse Formatduckdb.org
- Airflow Architecture: Key Components & Best Practiceshevodata.com
- SQL style guide by Simon Holywellsqlstyle.guide
- How the Open Knowledge Format can improve data sharing | Google Cloud Blogcloud.google.com