Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes

On Wed, Dec 29, 2021 at 11:31 AM Andres Freund <andres(at)anarazel(dot)de> wrote:

> Hi,
>
> On 2021-12-27 16:40:28 -0800, SATYANARAYANA NARLAPURAM wrote:
> > > Yet another problem is that if we are in XlogInsert() that means we are
> > > holding the buffer locks on all the pages we have modified, so if we
> add a
> > > hook at that level which can make it wait then we would also block any
> of
> > > the read operations needed to read from those buffers. I haven't
> thought
> > > what could be better way to do this but this is certainly not good.
> > >
> >
> > Yes, this is a problem. The other approach is adding a hook at
> > XLogWrite/XLogFlush?
>
> That's pretty much the same - XLogInsert() can trigger an
> XLogWrite()/Flush().
>
> I think it's a complete no-go to add throttling to these places. It's quite
> possible that it'd cause new deadlocks, and it's almost guaranteed to have
> unintended consequences (e.g. replication falling back further because
> XLogFlush() is being throttled).
>
> I also don't think it's a sane thing to add hooks to these places. It's
> complicated enough as-is, adding the chance for random other things to
> happen
> during such crucial operations will make it even harder to maintain.
>

Andres, thanks for the comments. Agreed on this based on the previous
discussions on this thread. Could you please share your thoughts on adding
it after SyncRepWaitForLSN()?

>
> Greetings,
>
> Andres Freund
>