✳flâneur — a map of the web's best reading

Some Thoughts on Rethinking Databases for Computational Science | Why (and How) Things Work

alabidan.blog · 834 words · saved by 1 readers

From 2014-2015, I was a Database Kernel Engineer in the Distributed Systems team at MongoDB. The team was responsible for designing and implementing protocols for executing database queries on data that is distributed across multiple machines. The query plan was automatically decided based on several factors (including read/write throughput, data locality, and data distribution). A shard key (i.e., a single indexed field or multiple fields) was used to distribute data into multiple chunks on different servers. Thus, the choice of a shard key could lead to different data distributions. This choice is especially important throughout the lifetime of executing different queries on the same data. As a result, domain knowledge of the data distribution and the lifetime of possible queries could be important in the query plan execution. In specific scientific fields (e.g., quantum physics), the data generated can be stored in a flat view. But this view does not take advantage of the data gener

From 2014-2015, I was a Database Kernel Engineer in the Distributed Systems team at MongoDB. The team was responsible for designing and implementing protocols for executing database queries on data that is distributed across multiple machines. The query plan was automatically decided based on several factors (including read/write throughput, data locality, and data distribution). A shard key (i.e., a single indexed field or multiple fields) was used to distribute data into multiple chunks on different servers. Thus, the choice of a shard key could lead to different data distributions. This cho

Explore this link on the map →

related reading