Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes

On Thu, Jan 6, 2022 at 11:27 AM SATYANARAYANA NARLAPURAM
<satyanarlapuram(at)gmail(dot)com> wrote:

> On Wed, Jan 5, 2022 at 9:46 AM Andres Freund <andres(at)anarazel(dot)de> wrote:
>>
>> Hi,
>>
>> On 2021-12-29 11:31:51 -0800, Andres Freund wrote:
>> > That's pretty much the same - XLogInsert() can trigger an
>> > XLogWrite()/Flush().
>> >
>> > I think it's a complete no-go to add throttling to these places. It's quite
>> > possible that it'd cause new deadlocks, and it's almost guaranteed to have
>> > unintended consequences (e.g. replication falling back further because
>> > XLogFlush() is being throttled).
>>
>> I thought of another way to implement this feature. What if we checked the
>> current distance somewhere within XLogInsert(), but only set
>> InterruptPending=true, XLogDelayPending=true. Then in ProcessInterrupts() we
>> check if XLogDelayPending is true and sleep the appropriate time.
>>
>> That way the sleep doesn't happen with important locks held / within a
>> critical section, but we still delay close to where we went over the maximum
>> lag. And the overhead should be fairly minimal.
>
>
> +1 to the idea, this way we can fairly throttle large and smaller transactions the same way. I will work on this model and share the patch. Please note that the lock contention still exists in this case.

Generally while checking for the interrupt we should not be holding
any lwlock that means we don't have the risk of holding any buffer
locks, so any other reader can continue to read from those buffers.
We will only be holding some heavyweight locks like relation/tuple
lock but that will not impact anyone except the writers trying to
update the same tuple or the DDL who want to modify the table
definition so I don't think we have any issue here because anyway we
don't want other writers to continue.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com