Exploring versions of the Postgres logical replication protocol
Logical Replication is one of the many ways a Postgres database can replicate data to other Postgres database (a.k.a standby). Logical replication directly reads from the write-ahead log (WAL), recording every database change, avoiding the need to intercept queries or periodically read the table. These changes are filtered, serialized and then sent to the standby servers where they can be applied. While logical replication is intended to be used by Postgres databases to send and receive changes, it also allows ETL tools like PeerDB to get a reliable stream of changes that can be processed as needed. Logical replication started by only allowing streaming of committed transactions. It then evolved to support in-flight transactions followed by two-phase commits and then parallel apply of in-flight transactions. This blog will dive into this evolution, its impact on performance, and present some useful benchmarks. This blog is useful for anyone who uses Postgres Logical Replication in prac
Introduction Logical Replication is one of the many ways a Postgres database can replicate data to other Postgres database (a.k.a standby). Logical replication directly reads from the write-ahead log (WAL), recording every database change, avoiding the need to intercept queries or periodically read the table. These changes are filtered, serialized and then sent to the standby servers where they can be applied. While logical replication is intended to be used by Postgres databases to send and receive changes, it also allows ETL tools like PeerDB to get a reliable stream of changes that can…
saved by
related reading
- PostgreSQL: Documentation: 18: 54.5. Logical Streaming Replication Protocolpostgresql.org
- Overcoming Pitfalls of Postgres Logical Decodingblog.peerdb.io
- Patching the PostgreSQL JDBC Driverengineering.zalando.com
- Ingest data from self-hosted PostgreSQL | Materialize Documentationmaterialize.com
- Schedule | POSETTE: An Event for Postgres 2024 - Citus Datacitusdata.com
- 67.4. Two-Phase Transactionspostgresql.org
- PostgreSQL: Documentation: 18: 54.1. Overviewpostgresql.org
- Postgres WAL Files and Sequence Numbers | Crunchy Data Blogcrunchydata.com
- 28.3. Write-Ahead Logging (WAL)postgresql.org
- Flink CDC for Postgres: Lessons Learned | sap1ens.comsap1ens.com
- PostgreSQL CDC - RisingWavedocs.risingwave.com
- 8.20. pg_lsn Typepostgresql.org