| From: | Sehrope Sarkuni <sehrope(at)jackdb(dot)com> |
|---|---|
| To: | Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com> |
| Cc: | Michael Paquier <michael(at)paquier(dot)xyz>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: [PATCH] Speed up pg_waldump TAP test and fix some GitHub CI Windows flakiness |
| Date: | 2026-09-01 14:04:13 |
| Message-ID: | CAH7T-aoLew8-5raz5c2UiUO29htvhhhgx_Ft1xQMwiZRdAr4bQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Thanks for the reviews!
I had put together a branch with some of feedback (comment clean up,
explicit full run through, etc). But seems like it'll be better to
split it up to allow for some backpatching so scrapped that and
went with the attached.
For the Windows testing I'm stuck with GitHub CI so it's kind of
a pain to cycle through it. Been using the Linux runtimes, total bytes,
and decode counts as a proxy to get some faster feedback.
Latest split out patches are attached. The first one does just the
`--limit=...` where we are only looking for one row. The second one
keeps track of the range of LSN to have smarter start / stop positions.
The third one goes a step further moves some of the bigger pieces out
of the loop so they are not repeated. The comments are cleaned up
as well.
Here's the timings / counts running on Linux:
time stdout decoded per decoded
(mean) via IPC archive scenario total
master 4.46s 139.1 MB 109.8 MB 393.5 MB
patch1 3.70s 57.7 MB 91.1 MB 273.4 MB
patch2 3.33s 37.1 MB 42.6 MB 127.8 MB
patch3 2.51s 19.4 MB 10.8 MB 64.2 MB
Each row includes the patches above it. Patch 1 caps the probe
runs via --limit, patch 2 bounds the scans with marker LSNs,
patch 3 decodes once per check instead of once per scenario.
Running in GitHub CI the Windows runtime twice for those are:
Part 1 - 769s, 247s
Part 2 - 128s, 95s
Part 3 - 121s, 178s
It's a _lot_ of noise in CI, but I think 1 + 2 is already the bulk of the gains.
Look like that's a bit further than Nazir's version as it picks a more
specific LSN for the ranges.
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/
| Attachment | Content-Type | Size |
|---|---|---|
| v3-0001-Stop-pg_waldump-TAP-test-probes-after-the-first-r.patch | text/x-patch | 2.0 KB |
| v3-0003-Decode-WAL-once-per-check-in-the-pg_waldump-TAP-t.patch | text/x-patch | 6.4 KB |
| v3-0002-Bound-pg_waldump-TAP-test-decodes-to-the-records-.patch | text/x-patch | 4.7 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Matthias van de Meent | 2026-09-01 14:30:12 | Re: Reducing relcache memory usage: deduping index shapes |
| Previous Message | Jan Nidzwetzki | 2026-09-01 14:01:06 | Re: Enable partitionwise join for partition keys wrapped by RelabelType |