✳flâneur — a map of the web's best reading

The Insatiable Postgres Replication Slot - Gunnar Morling

morling.dev · 1,552 words · saved by 1 readers

While working on a demo for processing change events from Postgres with Apache Flink, I noticed an interesting phenomenon: A Postgres database which I had set up for that demo on Amazon RDS, ran out of disk space. The machine had a disk size of 200 GiB which was fully used up in the course of less than two weeks. Now a common cause for this kind of issue are replication slots which are not advanced: in that case, Postgres will hold on to all WAL segments after the latest log sequence number (LSN) which was confirmed for that slot. Indeed I had set up a replication slot (via the Decodable CDC source connector for Postgres, which is based on Debezium). I then had stopped that connector, causing the slot to become inactive. The problem was though that I was really sure that there was no traffic in that database whatsoever! What could cause a WAL growth of ~18 GB/day then? What follows is a quick write-up of my investigations, mostly as a reference for my future self, but I hope this will

The Insatiable Postgres Replication Slot - Gunnar Morling Gunnar Morling Random Musings on All Things Software Engineering The Insatiable Postgres Replication Slot Posted at Nov 30, 2022 postgres cdc troubleshooting Table of Contents The Observation The Solution Take Away While working on a demo for processing change events from Postgres with Apache Flink, I noticed an interesting phenomenon: A Postgres database which I had set up for that demo on Amazon RDS, ran out of disk space. The machine had a disk size of 200 GiB which was fully used up in the course of less than two weeks. Now a common

Explore this link on the map →

related reading