Re: Sending unflushed WAL in physical replication

Hi,

> > Please find attached a POC patch that introduces changes to the WAL
> sender and
> > receiver, allowing WAL records to be sent to standbys before they are
> flushed
> > to disk on the primary during physical replication. [..]
>
> I didn't look at the code but your description of the design sounds OK.
>
>
Thanks for looking into it.

> I wanted to clarify: what happens if master doesn't increase flushPtr
> and replica runs out of memory for WAL records?
>

This is a great question. I'm currently working on implementing a solution
for this.
One possible solution is to write the records to a spill file when the
flush pointer
indicates that none have been flushed on the primary. Once they have been
flushed
on the primary, the records can then be copied from the spill file to the
WAL segments.
While this method may lead to increased I/O, if such spills are infrequent,
the overall
performance impact should be minimal.

Another option would be to notify the sender that there is no more space
available
and to pause sending additional data until records are flushed on the
sender side.
However, this approach could reintroduce some of the replication lag or
network
latency that we are aiming to minimize.

Kindly let me know your views.

Thank you,
Rahila Syed