Re: archive status ".ready" files may be created too early

From: "Bossart, Nathan" <bossartn(at)amazon(dot)com>
To: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
Cc: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "x4mmm(at)yandex-team(dot)ru" <x4mmm(at)yandex-team(dot)ru>, "a(dot)lubennikova(at)postgrespro(dot)ru" <a(dot)lubennikova(at)postgrespro(dot)ru>, "hlinnaka(at)iki(dot)fi" <hlinnaka(at)iki(dot)fi>, "matsumura(dot)ryo(at)fujitsu(dot)com" <matsumura(dot)ryo(at)fujitsu(dot)com>, "masao(dot)fujii(at)gmail(dot)com" <masao(dot)fujii(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: archive status ".ready" files may be created too early
Date: 2021-07-31 17:53:02
Message-ID: 997F25DB-5D57-4FB1-BA97-6110D8BF1415@amazon.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 7/31/21, 9:12 AM, "Alvaro Herrera" <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
> On 2021-Jul-31, Bossart, Nathan wrote:
>
>> On 7/30/21, 4:52 PM, "Alvaro Herrera" <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
>
>> > I noticed that XLogCtl->lastNotifiedSeg is protected by both the
>> > info_lck and ArchNotifyLock. I think it it's going to be protected by
>> > the lwlock, then we should drop the use of the spinlock.
>>
>> That seems reasonable to me. This means that the lock is acquired at
>> the end of every XLogWrite(),
>
> Uhh, actually that there sounds really bad; it's going to be a serious
> contention point.
>
> Another option might be to make it an atomic.

This is why I was trying to get away with just using info_lck for
reading lastNotifiedSeg. ArchNotifyLock is mostly intended to protect
RecordBoundaryMap. However, since lastNotifiedSeg is used in
functions like GetLatestRecordBoundarySegment() that access the map, I
found it easier to reason about things if I knew that it wouldn't
change as long as I held ArchNotifyLock.

I think the main downside of making lastNotifiedSeg an atomic is that
the value we first read in NotifySegmentsReadyForArchive() might not
be up-to-date, which means we might hold off creating .ready files
longer than necessary.

Nathan

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2021-07-31 19:33:36 Re: Split xlog.c
Previous Message vignesh C 2021-07-31 17:05:02 Re: [HACKERS] logical decoding of two-phase transactions