Re: [WIP] Pipelined Recovery

Hi Nitin, Imran,

On Mon, Aug 17, 2026 at 8:08 PM Nitin Jadhav
<nitinjadhavpostgres(at)gmail(dot)com> wrote:
>
> Hi,
>
> +1 on the overall direction. Decoupling WAL decode from redo looks
> like a promising way to improve CPU-bound recovery, and the
> performance numbers are encouraging. Thanks for posting v6; I reviewed
> the series at a high level.
>
> My main concern is the startup-to-producer state handoff. Startup
> exports a broad recovery snapshot, but the producer appears to restore
> only part of it. That can leave the two sides operating with different
> views of the recovery state.
>
> This becomes more important because the state is spread across
> startup-local state, producer-local state, WalPipelineParams, and
> shared mutable state. In this model, correctness depends on a clear
> ownership contract for each field: who writes it, who reads it, when
> it may change, and which copy is authoritative if values diverge.
> Without that contract, it is difficult to reason about behavior under
> races.

I share the concern here. It might be simpler to make the startup
process the owner of recovery policy and make the producer a bounded,
policy-free WAL reader. It might not be viable to decouple them
cleanly since decoding is what discovers edege cases such as
incomplete WAL, missing continuation records, and read boundaries.
Still wondering whether those conditions can be reported to startup as
events rather than letting the producer making these policy decisions.

> Relatedly, it would help to document queue/replay invariants in one
> place: strict LSN ordering, no silent drop/skip, defined ordering
> between data and shutdown/error messages, and clear behavior on
> queue-full and early-exit paths. The consumer side in-place
> deserialization path also needs an explicit memory-lifetime guarantee,
> so pointer validity during redo is unambiguous.

--
Regards,
Xuneng Zhou
HighGo Software Co., Ltd.