Re: Logical replication keepalive flood

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Peter Smith <smithpb2250(at)gmail(dot)com>
Cc: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Abbas Butt <abbas(dot)butt(at)enterprisedb(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Zahid Iqbal <zahid(dot)iqbal(at)enterprisedb(dot)com>
Subject: Re: Logical replication keepalive flood
Date: 2021-09-17 05:05:27
Message-ID: CAA4eK1JcFVay-O3PZnBFJMomYozJSiq1vzzYZQtTDE91JTH92w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Aug 12, 2021 at 8:03 AM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
>
> That base data is showing there are similar numbers of keepalives sent
> as there are calls made to WalSndWaitForWal. IIUC it means that mostly
> the loop is sending the special keepalives on the *first* iteration,
> but by the time of the *second* iteration the ProcessRepliesIfAny()
> will have some status already received, and so mostly sending another
> keepalive will be deemed unnecessary.
>
> Based on this, our idea was to simply skip sending the
> WalSndKeepalive(false) for the FIRST iteration of the loop only! PSA
> the patch 0002 which does this skip.
>

I think we should also keep in mind that there are cases where it
seems we are not able to send keep-alive at the appropriate frequency.
See the discussion [1]. This is to ensure that we shouldn't
unintentionally hamper some other workloads by fixing the workload
being discussed here.

[1] - https://www.postgresql.org/message-id/20210913.103107.813489310351696839.horikyota.ntt%40gmail.com

--
With Regards,
Amit Kapila.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message houzj.fnst@fujitsu.com 2021-09-17 05:08:36 RE: Column Filtering in Logical Replication
Previous Message Amit Kapila 2021-09-17 04:48:11 Re: walsender timeout on logical replication set