Re: archive status ".ready" files may be created too early

From: "Bossart, Nathan" <bossartn(at)amazon(dot)com>
To: "alvherre(at)alvh(dot)no-ip(dot)org" <alvherre(at)alvh(dot)no-ip(dot)org>
Cc: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "x4mmm(at)yandex-team(dot)ru" <x4mmm(at)yandex-team(dot)ru>, "a(dot)lubennikova(at)postgrespro(dot)ru" <a(dot)lubennikova(at)postgrespro(dot)ru>, "hlinnaka(at)iki(dot)fi" <hlinnaka(at)iki(dot)fi>, "matsumura(dot)ryo(at)fujitsu(dot)com" <matsumura(dot)ryo(at)fujitsu(dot)com>, "masao(dot)fujii(at)gmail(dot)com" <masao(dot)fujii(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: archive status ".ready" files may be created too early
Date: 2021-08-19 02:46:19
Message-ID: E65B47AA-C50B-4AD7-A90B-25D106582EB8@amazon.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 8/18/21, 4:47 PM, "alvherre(at)alvh(dot)no-ip(dot)org" <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
> On 2021-Aug-18, Bossart, Nathan wrote:
>> In the attached patch, I modified XLogInsertRecord() to simply set the
>> latch if we detect that flushRecPtr has advanced.
>
> Right, that's what I was thinking. I modified that slightly to use
> LogwrtResult.Flush (which should be fresh enough) instead of calling
> GetFlushRecPtr again, which saves a bit. I also changed it to > instead
> of >=, because if I understand you correctly we only care to notify if
> the flush pointer advanced, not in the case it stayed the same.

My thinking was that we needed to read flushRecPtr after registering
the boundary in case it advanced just before registration. And I used
>= because if flushRecPtr points to the end of the record, we should
be able to create the .ready file for the segment.

We can avoid acquiring the spinlock an extra time if we move the first
part of the cross-segment logic to before we update the local copy of
LogwrtResult. I attached a new version of the patch that does this.

The rest looks good to me.

Nathan

Attachment Content-Type Size
v11-0001-Avoid-creating-archive-status-.ready-files-too-e.patch application/octet-stream 21.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2021-08-19 03:08:57 Re: strange case of "if ((a & b))"
Previous Message Julien Rouhaud 2021-08-19 02:32:15 Re: support for windows robocopy in archive_command and restore_command