Re: [PATCH] Speed up pg_waldump TAP test and fix some GitHub CI Windows flakiness

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Sehrope Sarkuni <sehrope(at)jackdb(dot)com>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] Speed up pg_waldump TAP test and fix some GitHub CI Windows flakiness
Date: 2026-08-31 07:52:55
Message-ID: apUy1_dt86Rbaf7U@paquier.xyz
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Aug 28, 2026 at 11:43:54PM -0400, Sehrope Sarkuni wrote:
> Was testing a rebased Windows patch using GitHub CI and had some flaky slow
> test timeout errors for an unrelated pg_waldump test. On Windows (and only
> Windows) it was running for 500+ seconds and timing out. After a bit of
> research, I figured it's because of the notoriously slow spoofed IPC that
> perl does for the forked pg_waldump invocations (that then gets grepped by
> perl).

Your simple patch is also proving to make my runs of this test sort of
40% faster on Linux, at quick glance.

The --limit=1 additions are no-brainers in the commands where we don't
expect specific outputs and/or records. Same thing can be said for
the few --limit=5 cases, where I guess you've just chosen a number to
be representative enough for the output. This number looks fine here.

- '--start' => $start_lsn
+ '--start' => $contrecord_lsn

Hmm. Ok here. That's a nice trick. Perhaps this should have a
comment explaining why not the start_lsn but the contrecord LSN
because it's cheaper with less records?

- my @lines = test_pg_waldump($path, $start_lsn, $end_lsn);
- is(grep(!/^rmgr: \w/, @lines), 0, 'all output lines are rmgr lines');
-
- @lines = test_pg_waldump($path, $contrecord_lsn, $end_lsn);
+ my @lines = test_pg_waldump($path, $contrecord_lsn, $end_lsn);

Does this removal imply a loss of coverage?

+ @lines = test_pg_waldump($path, $start_lsn, $end_lsn,
'--fullpage', '--limit' => 5);

A limit of 5 for the fullpage case should be OK. Why not.

+ # --limit must come before a bare --stats: src/port/getopt_long.c,
+ # used on Windows, skips the argument after an optional_argument
+ # option given without "=". Drop the ordering once that is fixed.

Or fix the order here, without the comment. You have created already
a thread for the other issue with the options, let's just make this
code clean from the start so as we don't come back to it.

For the last case with "only lines for selected block", perhaps raise
a bit the limit for more output?
--
Michael

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2026-08-31 07:57:36 Re: pg_restore_attribute_stats() accepts non-finite values
Previous Message Michael Paquier 2026-08-31 07:38:07 Re: WAIT FOR command should do some query jumbling