Re: archive status ".ready" files may be created too early

From: "alvherre(at)alvh(dot)no-ip(dot)org" <alvherre(at)alvh(dot)no-ip(dot)org>
To: "Bossart, Nathan" <bossartn(at)amazon(dot)com>
Cc: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "x4mmm(at)yandex-team(dot)ru" <x4mmm(at)yandex-team(dot)ru>, "a(dot)lubennikova(at)postgrespro(dot)ru" <a(dot)lubennikova(at)postgrespro(dot)ru>, "hlinnaka(at)iki(dot)fi" <hlinnaka(at)iki(dot)fi>, "matsumura(dot)ryo(at)fujitsu(dot)com" <matsumura(dot)ryo(at)fujitsu(dot)com>, "masao(dot)fujii(at)gmail(dot)com" <masao(dot)fujii(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: archive status ".ready" files may be created too early
Date: 2021-08-18 17:40:03
Message-ID: 202108181740.zmdxsi3uci2k@alvherre.pgsql
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2021-Aug-18, Bossart, Nathan wrote:

> On 8/18/21, 10:06 AM, "alvherre(at)alvh(dot)no-ip(dot)org" <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
> > So that comment suggests that we should give the responsibility to bgwriter.
> > This seems good enough to me. I suppose if bgwriter has a long run of
> > buffers to write it could take a little bit of time (a few hundred
> > milliseconds?) but I think that should be okay.
>
> Do you think bgwriter should be the only caller of
> NotifySegmentsReadyForArchive(), or should we still have XLogWrite()
> call it?

I think XLogWrite should absolutely be the primary caller. The one in
bgwriter should be a backstop for the case you describe where the flush
pointer advanced past the registration point in XLogInsertRecord.

I realize this means there's a contradiction with my previous argument,
in that synchronous transaction commit calls XLogWrite at some point, so
we *are* putting the client-connected backend in charge of creating the
notify files. However, that only happens on transaction commit, where
we already accept responsibility for the WAL flush, not on each
individual XLOG record insert; also, the WAL writer will take care of it
sometimes, for transactions that are long-enough lived.

--
Álvaro Herrera Valdivia, Chile — https://www.EnterpriseDB.com/
"How strange it is to find the words "Perl" and "saner" in such close
proximity, with no apparent sense of irony. I doubt that Larry himself
could have managed it." (ncm, http://lwn.net/Articles/174769/)

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message alvherre@alvh.no-ip.org 2021-08-18 17:45:23 Re: archive status ".ready" files may be created too early
Previous Message Bossart, Nathan 2021-08-18 17:24:38 Re: archive status ".ready" files may be created too early