Re: Replication & recovery_min_apply_delay

From: Asim R P <apraveen(at)pivotal(dot)io>
To: Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Konstantin Knizhnik <knizhnik(at)garret(dot)ru>, Hao Wu <hawu(at)pivotal(dot)io>
Subject: Re: Replication & recovery_min_apply_delay
Date: 2020-03-16 12:00:00
Message-ID: CANXE4TeinQdw+M2Or0kTR24eRgWCOg479N8=gRvj9Ouki-tZFg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

A key challenge here is how to determine the starting point for WAL
receiver when the startup process starts it while still replaying WAL
that's already received. Hao and I wrote a much faster and less intrusive
solution to determine the starting point. Scan the first page of each WAL
segment file, starting from the one that's currently being read for
replay. If the first page is found valid, move to the next segment file
and check its first page. Continue this one segment file at a time until
either the segment file doesn't exist or the page is not valid. The
starting point is then the previous segment's start address.

There is no need to read till the end of WAL, one record at a time, like
the original proposal upthread did. The most recently flushed LSN does not
need to be persisted on disk.

Please see attached patch which also contains a TAP test to induce replay
lag and validate the fix.

Asim

Attachment Content-Type Size
v3-0001-Start-WAL-receiver-before-startup-process-replays.patch application/octet-stream 17.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Steele 2020-03-16 12:24:23 Re: [PATCH] Connection time for \conninfo
Previous Message Alexander Korotkov 2020-03-16 11:54:43 Re: pgsql: Add kqueue(2) support to the WaitEventSet API.