Re: Making pg_rewind faster

On Wed, Oct 15, 2025 at 7:56 PM Srinath Reddy Sadipiralla <
srinath2133(at)gmail(dot)com> wrote:

>
> thanks for updating, i have attached a diff patch on
> top of v9-0001 patch , where i tried to add more tests
> to improve the validation that we copy the WAL file even
> when it exists on both source and target but the size differs.
> I also tried to rename some variables for readability,while
> testing this out i found that usleep(1000); is not enough
> to show that there has been a copy which changed the
> stat->mtime of the file because i think the copy was very
> fast (less than 1 millisecond) so the mtime of the
> file after rewind's copy didn't change or filesystem precision
> didn't caught the change, the copying of the file is confirmed
> because the same file has different size before rewind and
> same after rewind which these tests proved
>

>
ok($corrupt_wal_seg_stat_before_rewind->size !=
> $corrupt_wal_seg_source_stat->size,"WAL segment $corrupt_wal_seg has
> different size in source vs target before rewind");
> ok 11 - WAL segment 000000010000000000000004 has different size in source
> vs target before rewind
>
> ok($corrupt_wal_seg_stat_after_rewind->size ==
> $corrupt_wal_seg_source_stat->size, "WAL segment $corrupt_wal_seg was
> copied: file sizes are same between target and source after rewind");
> ok 12 - WAL segment 000000010000000000000004 was copied: file sizes are
> same between target and source after rewind
>
> and more over the pg_rewind confirmed this with a new debug
> message which I added in this diff patch
>
> diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c
> index 69d7728ce44..743484f4833 100644
> --- a/src/bin/pg_rewind/filemap.c
> +++ b/src/bin/pg_rewind/filemap.c
> @@ -732,11 +732,11 @@ decide_wal_file_action(const char *fname, XLogSegNo
> last_common_segno,
> */
> if (file_segno < last_common_segno && source_size == target_size)
> {
> pg_log_debug("WAL segment \"%s\" not copied to target", fname);
> return FILE_ACTION_NONE;
> }
>
> + pg_log_debug("WAL segment \"%s\" is copied to target", fname);
> return FILE_ACTION_COPY;
> }
> ok 6 - run pg_rewind stderr /(?^:WAL segment \"000000010000000000000004\"
> is copied to target)/
>

>

btw i forgot to let you know that i have increased the
usleep(1000) to usleep(1000000) for the above reason
which helped to show the difference in mtime after rewind
adding the below extra validation that the file has been copied from
source to target ,this is already done in the v9-0001.diff patch.

ok($corrupt_wal_seg_stat_after_rewind->mtime >
$corrupt_wal_seg_stat_before_rewind->mtime,"WAL segment $corrupt_wal_seg
was copied: mtime after rewind > mtime before rewind");
ok 12 - WAL segment 000000010000000000000004 was copied: mtime after
rewind > mtime before rewind

On Fri, Oct 10, 2025 at 3:18 AM John H <johnhyvr(at)gmail(dot)com> wrote:

> Hey Srinath
>
> On Thu, Oct 9, 2025 at 12:09 PM Srinath Reddy Sadipiralla
> <srinath2133(at)gmail(dot)com> wrote:
> > ...
> > 1) if the timeline switch happens in middle of segment ,copy data from
> the last WAL segment and create WAL file with same segno but different
> timelineID,in this case the starting WAL file for the new timeline will be
> 000000020000000000000003
> > 2) if the timeline switch happens at segment boundary , just create next
> segment for this case the starting WAL file for the new timeline will be
> 000000020000000000000004
> >
> > so basically the files which exists in source and not in target like the
> new timeline WAL segments will be copied to target in total before we reach
> decide_wal_file_action , so i think we don't need to think about copying
> WAL files after divergence point by calculating and checking against
> last_common_segno which we are doing in our current approach , i think we
> can just do
> >
> > ...
>
> That's a great point. I need to think about it some more but the
> reasoning makes sense to me.
> I think 'last_common_segno ' is only useful as another sanity check
> but we already have the size
> ones.
>

it would be awesome to know ,if you have any update on this.

--
Thanks,
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/