Dean Lau
0 followers · 271 views
on the atlas · 3
- Spark Architecture: A Deep Dive. Apache Spark is an open-source… | by Amit Joshi | Medium1 savers
- Optimizing RAG: A Guide to Choosing the Right Vector Database | by Mutahar Ali | Medium1 savers
- Taints and Tolerations, Node Affinity, and Node Selector — Explained | by Ilan Spiegel | saas-infra | Medium1 savers
highlights · 8
A task is the smallest unit of work in Spark, representing a unit of computation that can be performed on a single partition of data.
Spark Architecture: A Deep Dive. Apache Spark is an open-source… | by Amit Joshi | MediumSparkContext is the entry point for any Spark functionality.
Spark Architecture: A Deep Dive. Apache Spark is an open-source… | by Amit Joshi | MediumThe driver is the program or process responsible for coordinating the execution of the Spark application.
Spark Architecture: A Deep Dive. Apache Spark is an open-source… | by Amit Joshi | MediumExecutors are worker processes responsible for executing tasks in Spark applications.
Spark Architecture: A Deep Dive. Apache Spark is an open-source… | by Amit Joshi | MediumThe cluster manager is responsible for allocating resources and managing the cluster on which the Spark application runs.
Spark Architecture: A Deep Dive. Apache Spark is an open-source… | by Amit Joshi | Mediumthe maximum vector dimension supported by pgvector (2000) is quite restrictive in comparison to specialized vector databases like Weaviate, which can handle dimensions as high as 65535.
Optimizing RAG: A Guide to Choosing the Right Vector Database | by Mutahar Ali | Mediumvector libraries store are mostly used for static data, where the index data is immutable
Optimizing RAG: A Guide to Choosing the Right Vector Database | by Mutahar Ali | MediumTaint is a feature in Kubernetes that allows users to mark a node as "tainted" with a specific key and value. Pods that do not have toleration for the taint will not be scheduled on the tainted node.
Taints and Tolerations, Node Affinity, and Node Selector — Explained | by Ilan Spiegel | saas-infra | Medium