Re: Adding hook in BufferSync for backup purposes

From: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
To: Андрей Бородин <x4mmm(at)yandex-team(dot)ru>
Cc: pgsql-hackers(at)postgresql(dot)org, root(at)simply(dot)name
Subject: Re: Adding hook in BufferSync for backup purposes
Date: 2017-08-07 11:51:18
Message-ID: 20170807115118.ucpvxof57gs5vtae@alvherre.pgsql
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Андрей Бородин wrote:

> ==What==
> I propose to add hook inside BufferSync() function it inform extensions that we
> are going to write pages to disk. Please see patch attached. I pass a timestamp
> of the checkpoint, but it would be good if we could also pass there number of
> checkpoint or something like this to ensure that some checkpoints were not lost
> (this could yield malformed backups).
> ==State==
> This is just an idea to discuss, I could not find something like this in
> pgsql-hackers as for now. Neither I could find similar hooks in the code.
> Is this hook sufficient to implement page tracking for differential backups?
> I’m not sure, but seems like it is.


I remember discussing the topic of differential base-backups with
somebody (probably Marco and Gabriele). The idea we had was to have a
new relation fork which stores an LSN for each group of pages,
indicating the LSN of the newest change to those pages. The backup tool
"scans" the whole LSN fork, and grabs images of all pages that have LSNs
newer than the one used for the previous base backup.

(I think your sketch above should use LSNs rather than timestamps).

I suppose your hook idea lets you implement the LSN fork in an
extension, rather than having it be part of core. The idea of hooking
onto BufferSync makes me uneasy, though -- like it's not the correct
place to do it. I think it should be at the point where the buffer is
modified (i.e. when WAL is written) rather than when it's checkpointed

Álvaro Herrera
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Ashutosh Sharma 2017-08-07 12:20:05 Re: Page Scan Mode in Hash Index
Previous Message Amit Langote 2017-08-07 06:54:44 Re: expanding inheritance in partition bound order