Re: Proposing pg_hibernate

From: Gurjeet Singh <gurjeet(at)singh(dot)im>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Andres Freund <andres(at)2ndquadrant(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, PGSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Proposing pg_hibernate
Date: 2014-06-11 02:03:56
Message-ID: CABwTF4USCLYu2YxGwah7snUaJtprfOMEn7AqkTY2SzTtRd4Jog@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Jun 10, 2014 at 12:02 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> If recovery has been running for a long time, then restoring
> buffers from some save file created before that is probably a bad
> idea, regardless of whether the buffers already loaded were read in by
> recovery itself or by queries running on the system. But if you're
> saying that doesn't happen, then there's no problem there.

Normally, it won't happen. There's one case I can think of, which has
to coincide with a small window of time for such a thing to happen.

Consider this:
.) A database is shutdown, which creates the save-files in
$PGDATA/pg_hibernator/.
.) The database is restarted.
.) BlockReaders begin to read and restore the disk blocks into buffers.
.) Before the BlockReaders could finish*, a copy of the database is
taken (rsync/cp/FS-snapshot/etc.)
This causes the the save-files to be present in the copy, because
the BlockReaders haven't deleted them, yet.
* (The BlockReaders ideally finish their task in first few minutes
after first of them is started.)
.) The copy of the database is used to restore and erect a warm-standby.
.) The warm-standby starts replaying logs from WAL archive/stream.
.) Some time (hours/weeks/months) later, the warm-standby is promoted
to be a master.
.) It starts the Postgres Hibernator, which sees save-files in
$PGDATA/pg_hibernator/ and launches BlockReaders.

At this point, the BlockReaders will restore the blocks that were
present in original DB's shared-buffers at the time of shutdown. So,
this would fetch blocks into shared-buffers that may be completely
unrelated to the blocks recently operated on by the recovery process.

And it's probably accepted by now that such a bahviour is not
catastrophic, merely inconvenient.

Best regards,
--
Gurjeet Singh http://gurjeet.singh.im/

EDB www.EnterpriseDB.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Dunstan 2014-06-11 02:11:15 Re: "RETURNING PRIMARY KEY" syntax extension
Previous Message Tom Dunstan 2014-06-11 02:01:02 Re: "RETURNING PRIMARY KEY" syntax extension