| From: | Srinath Reddy Sadipiralla <srinath2133(at)gmail(dot)com> |
|---|---|
| To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
| Cc: | hlinnaka(at)iki(dot)fi, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
| Subject: | Re: pg_rewind: Remove recovery at the start of rewind |
| Date: | 2026-11-12 16:28:04 |
| Message-ID: | CAFC+b6p+8OhUPgdvJ7_j3g84tmFpGQ3Yia6im6H9FQUX0_+0bQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Rebased the patch.
On Sun, Aug 30, 2026 at 9:54 AM Srinath Reddy Sadipiralla <
srinath2133(at)gmail(dot)com> wrote:
> Hi,
>
> I have been exploring whether pg_rewind can avoid running
> crash recovery on a target that was stopped without a clean
> shutdown, as at the end of rewind changed rel blocks and non
> rel files are copied or removed wrt source.
>
> Currently, pg_rewind starts the target in single-user mode to
> complete crash recovery before rewind begins. This reads
> all WALs from the last checkpoint and does the replay.
>
> The attached POC instead scans target WAL directly.
> pg_rewind already scans target WAL from the last common
> checkpoint to construct the page map of blocks changed on
> the target. This patch extends that processing to locate the
> end of target WAL as well, so that pg_rewind can proceed
> without starting postgres on the target.
>
> The target is still required to be stopped. The patch rejects
> a live target, removes a stale postmaster.pid when appropriate,
> and removes --no-ensure-shutdown because pg_rewind no longer
> runs crash recovery.
>
> For locating the checkpoint preceding the divergence point, the normal
> path follows the WAL record chain backwards from the divergence point.
> The patch also has a fallback: if the record at the divergence point cannot
> be read, it scans forward from the target control-file checkpoint, provided
> that checkpoint precedes the divergence point.
>
> The motivation is not to avoid reading target WAL. Both the current
> implementation and the POC must read target WAL to build the page map.
> The expected saving is avoiding redo of target WAL into relation files
> that pg_rewind subsequently makes equal to the source, or removes because
> they exist only on the target.
>
> As a initial local benchmark, I compared current pg_rewind with this
> POC. The workload generated approximately the following amount of
> uncheckpointed target-side data:
> current POC reduction
> 100 MB 1.404 s 0.979 s 30.3%
> 500 MB 5.358 s 3.991 s 25.5%
> 1000 MB 10.473 s 7.097 s 32.2%
> 2000 MB 49.904 s 31.294 s 37.3%
> The result is a ~1.5x speedup in this workload, along with some tests
> I did, and also make check-world passes.
>
> would love to hear more thoughts on this.
>
>
> --
> Thanks :)
> Srinath Reddy Sadipiralla
> EDB: https://www.enterprisedb.com/
>
--
Thanks :)
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/
| Attachment | Content-Type | Size |
|---|---|---|
| v2-0001-pg_rewind-scan-an-unclean-target-s-WAL-instead-of-fo.patch | application/octet-stream | 26.8 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Bharath Rupireddy | 2026-11-12 17:04:35 | Re: Teach pg_upgrade to deal with invalid databases |
| Previous Message | Joao Rito | 2026-11-12 16:12:57 | Re: Hash index: stop registering pin-only bucket pages in WAL (remove REGBUF_NO_CHANGE) |