Re: Re: BUG #13685: Archiving while idle every archive_timeout with wal_level hot_standby

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Re: BUG #13685: Archiving while idle every archive_timeout with wal_level hot_standby
Date: 2015-11-06 07:47:25
Message-ID: CAB7nPqRUTRQvBYiDY-rC04+FOg0vUKCOAvAFZViw7fw8CnJjWA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs pgsql-hackers

On Thu, Nov 5, 2015 at 3:00 PM, Michael Paquier
<michael(dot)paquier(at)gmail(dot)com> wrote:
> On Wed, Nov 4, 2015 at 7:33 PM, Andres Freund wrote:
>> As soon as a single checkpoint ever happened the early-return logic in
>> CreateCheckPoint() will fail to take the LogStandbySnapshot() in
>> CreateCheckPoint() into account. The test is:
>> if (curInsert == ControlFile->checkPoint +
>> MAXALIGN(SizeOfXLogRecord + sizeof(CheckPoint)) &&
>> ControlFile->checkPoint == ControlFile->checkPointCopy.redo)
>> which obviously doesn't work if there's been a WAL record logged after
>> the redo pointer has been determined etc.
>
> Yes. If segment switches are enforced at a pace faster than
> checkpoint_timeout, this check considers that a checkpoint needs to
> happen because a SWITCH_XLOG record is in-between. I am a bit
> surprised that this should happen actually. The segment switch
> triggers a checkpoint record, and vice-versa, even for idle systems.
> Shouldn't we make this check a bit smarter then?

Ah, the documentation clearly explains that setting archive_timeout
will enforce a segment switch if any activity occurred, including a
checkpoint:
http://www.postgresql.org/docs/devel/static/runtime-config-wal.html#RUNTIME-CONFIG-WAL-ARCHIVING
I missed that, sorry.

I have as well thought a bit about adding a space-related constraint
on the standby snapshot generated by the bgwriter, so as to not rely
entirely on the interval of 15s. I finished with the attached that
uses a check based on CheckPointSegments / 8 to be sure that at least
this number of segments has been generated since the last checkpoint
before logging a new snapshot. I guess that's less brittle than the
last patch. Thoughts?
--
Michael

Attachment Content-Type Size
20151106_archive_idle_v3.patch text/x-patch 4.4 KB

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Haribabu Kommi 2015-11-06 09:34:05 Re: Memory leak with PL/Python trigger
Previous Message Michael Paquier 2015-11-06 06:53:47 Re: BUG #13741: vacuumdb does not accept valid password

Browse pgsql-hackers by date

  From Date Subject
Next Message Fabien COELHO 2015-11-06 07:48:06 Re: pgbench gaussian/exponential docs improvements
Previous Message Kouhei Kaigai 2015-11-06 07:02:48 Re: CustomScan support on readfuncs.c