Re: pg_recvlogical: Prevent flushed data from being re-sent after restarting replication

From: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
To: Mircea Cadariu <cadariu(dot)mircea(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: pg_recvlogical: Prevent flushed data from being re-sent after restarting replication
Date: 2026-01-07 03:36:32
Message-ID: CAHGQGwH2Xi9s9kpmG9xp+qKassjb+RqiEsoKZFOk0MsaKPX4ag@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Dec 29, 2025 at 9:45 PM Mircea Cadariu <cadariu(dot)mircea(at)gmail(dot)com> wrote:
>
> Hi,
>
> Thanks for the patch updates.
>
> On 26/12/2025 10:28, Fujii Masao wrote:
>
> Maybe it's better to use slurp_file(). We already have wait_for_log() to
> wait for a message in the cluster's log file, but there's no helper function
> to wait for specific content to appear in an arbitrary file.
>
> To support waiting for output in pg_recvlogical's output file,
> I added a new helper that uses slurp_file() (see the attached 0002 patch).
> I also updated the 0003 patch (the pg_recvlogical reconnection test) to
> use this helper instead of pg_read_file(). Thoughts?
>
> Agreed, nice addition.
>
> I applied the v3-000* patch set and it builds successfully and passes the tests on my laptop.
>
> However the CI seems not completely happy yet, with previous 2 runs not green for Windows. Could it be there's an issue with executing the test on Windows?

Thanks for the report!

The TAP test failed on Windows because it attempted to terminate
pg_recvlogical using a TERM signal, which isn't available there.
As a result, the test waited indefinitely for pg_recvlogical to exit
and finally timed out.

To address this, I updated the 0003 patch so that the test passes
--endpos to pg_recvlogical on Windows only. This allows pg_recvlogical
to terminate without signals, by generating WAL until the current
position reaches the specified end position. OTOH, on non-Windows
platforms, the test continues to use signals to terminate pg_recvlogical.

This approach may be somewhat unstable. If there's a more robust
way to terminate pg_recvlogical on Windows, I'd be happy to switch
to it, but I couldn't come up with a better option.

Updated patches are attached.

Regards,

--
Fujii Masao

Attachment Content-Type Size
v4-0004-pg_recvlogical-remove-unnecessary-OutputFsync-ret.patch application/octet-stream 2.6 KB
v4-0003-Add-test-for-pg_recvlogical-reconnection-behavior.patch application/octet-stream 4.2 KB
v4-0001-pg_recvlogical-Prevent-flushed-data-from-being-re.patch application/octet-stream 2.2 KB
v4-0002-Add-a-new-helper-function-wait_for_file-to-Utils..patch application/octet-stream 3.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Xueyu Gao 2026-01-07 03:41:29 [PATCH]remove extra blank line in the comment of pg_strxfrm_enabled()
Previous Message Srinath Reddy Sadipiralla 2026-01-07 03:34:11 Re: psql: Add tab completion for \copy pstdin and pstdout