Hieu Tran
3 followers · 2 following · 854 views
on the atlas · 31
- Incremental Database Computations2 savers
- sync package - sync - Go Packages1 savers
- Amazon SQS FAQs | Message Queuing Service | AWS1 savers
- Threading Model · React Native1 savers
- Ingest data from PostgreSQL CDC - RisingWave1 savers
- FROM clause - RisingWave1 savers
- Flink CDC for Postgres: Lessons Learned | sap1ens.com1 savers
- Debezium and TimescaleDB1 savers
- debezium-examples/postgres-toast at main · debezium/debezium-examples1 savers
- Schedule | POSETTE: An Event for Postgres 2024 - Citus Data1 savers
- Exploring versions of the Postgres logical replication protocol1 savers
- Go Slices: usage and internals - The Go Programming Language2 savers
- Overcoming Pitfalls of Postgres Logical Decoding1 savers
- Ingest data from self-hosted PostgreSQL | Materialize Documentation1 savers
- Patching the PostgreSQL JDBC Driver2 savers
- PostgreSQL: Documentation: 17: 66.4. Two-Phase Transactions1 savers
- PostgreSQL: Documentation: 17: 53.5. Logical Streaming Replication Protocol1 savers
- Is Software Architecture same as System Design? : r/softwarearchitecture1 savers
- PostgreSQL: Documentation: 17: 53.1. Overview1 savers
- Book Review: Modern Software Engineering1 savers
- Postgres WAL Files and Sequence Numbers | Crunchy Data Blog1 savers
- PostgreSQL: Documentation: 17: 8.20. pg_lsn Type1 savers
- PostgreSQL: Documentation: 17: 28.3. Write-Ahead Logging (WAL)1 savers
- Read Medium articles with AI2 savers
- Workflow definitions visuliazations - Community Support - Temporal1 savers
- High-Performance Personalization For Modern Frontends1 savers
- Graph Thinking. A kind of cognitive framework called… | by Paco Xander Nathan | Knowledge Technologies | Medium1 savers
- Demystifying function parameters in Go – Alex Edwards2 savers
- Making React fast by default and truly reactive1 savers
- Open Source Durable Execution | Temporal Technologies1 savers
- Curius / Onboarding2704 savers
highlights · 43
Q can be described in terms of operations on tables, Q^Δ can only be described in terms of operations on Z-sets.
Incremental Database ComputationsA dead-letter queue is an Amazon SQS queue to which a source queue can send messages if the source queue’s consumer application is unable to consume the messages successfully
Amazon SQS FAQs | Message Queuing Service | AWSYou can configure the Amazon SQS message retention period to a value from 1 minute to 14 days. The default is 4 days.
Amazon SQS FAQs | Message Queuing Service | AWSYou can configure the Amazon SQS message retention period to a value from 1 minute to 14 days.
Amazon SQS FAQs | Message Queuing Service | AWSAmazon SQS provides common middleware constructs such as dead-letter queues and poison-pill management
Amazon SQS FAQs | Message Queuing Service | AWSAmazon SNS allows applications to send time-critical messages to multiple subscribers through a “push” mechanism, eliminating the need to periodically check or “poll” for updates.
Amazon SQS FAQs | Message Queuing Service | AWSPostgreSQL CDC parameters. The data format is fixed as FORMAT PLAIN ENCODE JSON so it does not need to be specified.
Ingest data from PostgreSQL CDC - RisingWavemultiple sources are specified, the result is all the sources’ Cartesian product (i.e., cross join).
FROM clause - RisingWaveFlink CDC source always has parallelism of 1, so the ordering is unaffected.
Flink CDC for Postgres: Lessons Learned | sap1ens.comdepending on your checkpointing interval you should be ok with seeing some failed checkpoints until snapshotting is over, which could mean tweaking some monitors / alerts.
Flink CDC for Postgres: Lessons Learned | sap1ens.comFlink, you get solid fault tolerance, reliability and scalability guarantees
Flink CDC for Postgres: Lessons Learned | sap1ens.comContinuous aggregates provide automatic statistical calculations over data that is stored in hypertables
Debezium and TimescaleDBPostgreSQL has a hard limit on the page size. This means that values larger than ca. 8 KB need to be stored using TOAST storage
debezium-examples/postgres-toast at main · debezium/debezium-examplesLogical decoding is a single-threaded process, whereas Postgres allows multiple concurrent connections/threads to ingest data.
Overcoming Pitfalls of Postgres Logical DecodingIn Materialize, a cluster is an isolated environment, similar to a virtual warehouse in Snowflake
Ingest data from self-hosted PostgreSQL | Materialize Documentationfix involved updating the client to keep track of the LSN of the last Replication message received from the server and the LSN of the latest message confirmed by the client
Patching the PostgreSQL JDBC DriverTwo-phase transactions are intended for use by external transaction management systems. PostgreSQL follows the features and model proposed by the X/Open XA standard, but does not implement some less often used aspects.
PostgreSQL: Documentation: 17: 66.4. Two-Phase TransactionsTwo-phase transactions are intended for use by external transaction management systems. PostgreSQL follows the features and model proposed by the X/Open XA standard, but does not implement some less often used aspects.
PostgreSQL: Documentation: 17: 66.4. Two-Phase TransactionsVersion 3 is supported only for server version 15 and above, and it allows streaming of two-phase commits.
PostgreSQL: Documentation: 17: 53.5. Logical Streaming Replication Protocolsystem design can include more things than the Software architecture
Is Software Architecture same as System Design? : r/softwarearchitectureAll communication is through a stream of messages. The first byte of a message identifies the message type, and the next four bytes specify the length of the rest of the message (this length count includes itself, but not the message-type byte).
PostgreSQL: Documentation: 17: 53.1. Overviewour industry has struggled with the difference between programming and engineering
Book Review: Modern Software EngineeringThe WAL file name is in the format TTTTTTTTXXXXXXXXYYYYYYYY. Here 'T' is the timeline, 'X' is the high 32-bits from the LSN, and 'Y' is the low 32-bits of the LSN.
Postgres WAL Files and Sequence Numbers | Crunchy Data BlogThe pg_lsn data type can be used to store LSN (Log Sequence Number) data which is a pointer to a location in the WAL
PostgreSQL: Documentation: 17: 8.20. pg_lsn Typethe cost of syncing the WAL is much less than the cost of flushing the data pages
PostgreSQL: Documentation: 17: 28.3. Write-Ahead Logging (WAL)these steps are related to an earlier notion in computer science called stepwise refinement. This can also provide a structured way of developing notes for yourself and your team
Read Medium articles with AIMost workflow complexity is not in the sequencing of actions but in arguments and state management.
Workflow definitions visuliazations - Community Support - TemporalThis approach allows for incredibly granular personalization and is great for SEO because all the content is in the initial HTML. This approach even can work without an edge worker too
High-Performance Personalization For Modern FrontendsFor sense-making, experts leverage emergent patterns within graph structures.
Graph Thinking. A kind of cognitive framework called… | by Paco Xander Nathan | Knowledge Technologies | MediumGo runtime implements a map as a pointer to a runtime.hmap struct
Demystifying function parameters in Go – Alex EdwardsFun fact: the children prop is always different, so memo is useless for components with children.
Making React fast by default and truly reactiveOne of the most interesting pieces of tech I've seen in years… Temporal does to backend and infra, what React did to frontend… the surface exposed to the developer is a beautiful "render()" function to organize your backend workflows.
Open Source Durable Execution | Temporal Technologieswhen using binary parameters, PostgreSQL client sends only one request per query, but all query parameters are being sent as a binary.
PgBouncer and prepared statements | Yet Another Dev Blogif there are one hundred client connections, then one hundred PostgreSQL Server processes serve those one hundred client connections.
PgBouncer PostgreSQL performanceif there are one hundred client connections, then one hundred PostgreSQL Server processes serve those one hundred client connections.
PgBouncer PostgreSQL performanceA Bloom filter can guarantee the absence of an element from a set, but it can only give an estimation about its presence.
Bloom filter | Docsthe bitmap gets too large we convert it to "lossy" style, in which we only remember which pages contain matching tuples instead of remembering each tuple individually. When that happens, the table-visiting phase has to examine each tuple on the page and recheck
PostgreSQL: Re: Bitmap indexes etc.the source code is my favorite museum to exhibit the Rosetta Stone
Hieu Tran BlogDNS -> Load Balancer -> API gateway -> Backend service
dns - Load balancer and API Gateway confusion - Stack OverflowThe cache hit ratio on the TLB (Translation Lookaside Buffer) will be better (because memory is shared among multiple threads)
Postgres Architecture | Saurabh ThakurThe value should be set to 15% to 25% of the machine’s total RAM. For example: if your machine’s RAM size is 32 GB, then the recommended value for shared_buffers is 8 GB
How to tune PostgreSQL for memory | EDBhis is why we pay managed providers to be the wise experts who know how to tune shared_buffers and take backups without interrupting service
Migrating from Supabasewhile the port is advertised by the replica itself during the handshake, however the port may be wrong for the same reason as exposed in point 1.
High availability with Redis Sentinel | Docs