Re: Challenges with huge postgresql deployments in 300+ TBs

Hi Kavya,

> 3. Fixing a standby that lags a lot

Of the operational walls, this is the one I would size before
committing rather than after, so I put together a small harness that
turns it into a number: it loads indexed data, pauses replay to
separate WAL generation from replay, then times pure replay from local
WAL and reports MB/s with a 95% CI and a lag curve. About 15 minutes
on a couple hundred MB, and it refuses assert-enabled builds so the
timing is real:

https://github.com/Manuelreyesbravo/standby-replay-ceiling

The numbers are hardware-specific -- one laptop core replays around
190 MB/s here, CPU-bound. The point is to run it on a subset of your
intended storage and CPU and get your own ceiling before building the
full system, which matches Adrian's start-from-a-test-setup
suggestion; whether one replay core keeps up then depends on how much
WAL your write workload generates.

It measures that wall, it does not move it. Happy to adjust it if your
setup would test differently.

Regards,
Manu