| From: | Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com> |
|---|---|
| To: | surya poondla <suryapoondla4(at)gmail(dot)com> |
| Cc: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: pg_walinspect: add functions to locate and list WAL by time and LSN |
| Date: | 2026-10-09 01:28:50 |
| Message-ID: | 863CC649-43A8-4C68-A633-52583B75E490@gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
> On Oct 9, 2026, at 06:36, surya poondla <suryapoondla4(at)gmail(dot)com> wrote:
>
> Hi Chao,
>
> Thanks for v3. Moving to outer anchors with the fallback to the current WAL position addresses my concern with v2, and I appreciate the new tests
> covering both the empty-window case and the DELETE scenario.
>
> v3 applies cleanly to master (f884f359f5a) and builds without warnings.
> Locally, the regression tests and both TAP tests pass, and so does my earlier reproducer. The returned range now brackets the DELETE:
>
> start_lsn | end_lsn | delete_lsn
> ------------+------------+------------
> 0/017F0220 | 0/017F02E0 | 0/017F0280
>
>
> However, CFBot's macOS - Meson job fails on v3 (https://github.com/postgres/postgres-cfbot/actions/runs/37738280802/job/113183259446) in 001_timeline.pl, during the archive_mode=0 pass:
>
> ERROR: WAL segment needed for the requested range is missing
> DETAIL: Segment 4 is not present in pg_wal.
> ... FROM pg_get_wal_files('0/04001790', pg_current_wal_lsn())
>
> The standby's server log from that run points to a race in the test rather than a bug in pg_get_wal_files():
>
> 06:52:52.657 received promote request
> 06:52:52.701 checkpoint starting: force
> 06:52:52.938 checkpoint complete: ... 0 WAL file(s) added, 0 removed, 2 recycled
> 06:52:52.972 ERROR: WAL segment needed for the requested range is missing
>
> The checkpoint forced by the promotion recycled two WAL segments, and since segment 4, which holds start_lsn, was gone 34 ms later, it must have been
> one of them. On a faster machine the query runs before that checkpoint completes, which would explain why the test passes locally.
>
> Keeping that WAL on the standby makes the test deterministic. I confirmed this locally with a copy of the test that forces a CHECKPOINT right after
> promote(): it fails every time with the same "Segment 4 is not present" error, and passes once the standby has wal_keep_size set:
>
> $standby->init_from_backup($primary, $backup_name, has_streaming => 1);
> $standby->append_conf('postgresql.conf', "wal_keep_size = '64MB'");
>
> src/test/recovery/t/004_timeline_switch.pl uses wal_keep_size the same way.
Fixed by setting wal_keep_size, which prevents the required WAL file from being recycled.
>
> One small question on 0001. For a point lookup at the current LSN on a segment boundary, v3 now returns the preceding segment, which makes sense
> given that the next file may not exist yet. pg_walfile_name(), though, maps the same LSN to the next segment (it uses XLByteToSeg). So at that one LSN
> the two functions name different files. I'm fine with the choice, since the docs explain it, but it may be worth a sentence noting the difference from
> pg_walfile_name().
>
I think the two functions do different things, although their names sound similar. pg_walfile_name(lsn) performs a pure calculation that maps an LSN to a WAL file name without checking if the file exists, while pg_get_wal_files(start_lsn, end_lsn) returns a list of WAL files found on disk. We can add a sentence to the doc to clarify this difference.
PFA v4:
* 0001 - Fixed the fragile TAP test, and enhanced the doc.
* 0002 - Tiny doc formatting changes.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
| Attachment | Content-Type | Size |
|---|---|---|
| v4-0001-pg_walinspect-add-function-to-list-WAL-files-by-L.patch | application/octet-stream | 29.3 KB |
| v4-0002-pg_walinspect-add-function-to-locate-WAL-by-time.patch | application/octet-stream | 60.1 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Henson Choi | 2026-10-09 01:35:02 | Re: Row pattern recognition |
| Previous Message | zengxx | 2026-10-09 01:04:06 | Re: Skip a redundant singleton GROUP BY node |