pg_prefaulter: Solving Follower Lag and Restart times...

pg_prefaulter is used to mitigate the effects of PostgreSQL replication lag.

https://github.com/joyent/pg_prefaulter

If you’ve seen replication lag due to pg_repack, VACUUM, or a write-heavy
workload, the pg_prefaulter reduces the apply lag on followers.

In a primary-follower cluster using PostgreSQL streaming replication, the
PostgreSQL’s primary database instance enjoys the benefit of being able to
parallelize its random IO workload. The follower, however, is not so
fortunate. The follower receives a sequential stream of WAL records and
sequentially schedules all pread(2) IOs in order in a single process with
blocking IO. If the average IO latency per random pread(2) is ~8ms, that means
that the follower can only complete ~125 random IO pread(2) calls per second
(i.e. only 125 WAL records/second).

The pg_prefaulter reads the stream of WAL records using pg_xlogdump(1) and
schedules concurrent IOs in order to prefault PostgreSQL heap pages into the
follower’s filesystem cache. The effect of this is dramatic in that the follower
is able to apply pages with a filesystem cache-hit.

pg_prefaulter can also pull the WAL files from process arguments when a
connection is not available, this allows pg_prefaulter to accelerate the
recovery and startup time of PostgreSQL. If you’ve ever sat twiddling your
thumbs waiting for the database to start up, pg_prefaulter should improve the
startup time of PostgreSQL.

Intro: https://github.com/joyent/pg_prefaulter#pg_prefaulter

Background: https://github.com/joyent/pg_prefaulter#background

Cheers. -sc

--
Sean Chittenden