✳flâneur — a map of the web's best reading

Spark Architecture: A Deep Dive. Apache Spark is an open-source… | by Amit Joshi | Medium

medium.com · 1,029 words · saved by 1 readers

Apache Spark is an open-source distributed computing system designed for big data processing and analytics. Spark is known for its speed and efficiency. If you want more introduction about spark I have already covered that part in this blog. In this article, we will explore the following topics: Let's start... The Apache Spark framework uses a master-slave architecture that consists of a driver, which runs as a master node, and many executors that run across as worker nodes in the cluster. Apache Spark can be used for batch processing and real-time processing as well. Before understanding the Spark architecture let's understand the Applications of Spark Architecture which exists in the above diagram: The driver is the program or process responsible for coordinating the execution of the Spark application. It runs the main function and creates the SparkContext, which connects to the cluster manager. Executors are worker processes responsible for executing tasks in Spark applications. The

Spark Architecture: A Deep Dive Amit Joshi 5 min read · Jun 1, 2023 -- 4 Listen Share Press enter or click to view image in full size Apache Spark is an open-source distributed computing system designed for big data processing and analytics. Spark is known for its speed and efficiency. If you want more introduction about spark I have already covered that part in this blog. In this article, we will explore the following topics: spark Architecture And Applications working of spark Architecture Abstractions of Apache Spark Cluster Manager Types Execution Modes Let's start... The Apache Spark fram

Explore this link on the map →

saved by

related reading