Re: Hooks to track changed pages for backup purposes

Hi!
Thank you for your interest and experiment results.
> 13 сент. 2017 г., в 15:43, Ants Aasma <ants(dot)aasma(at)eesti(dot)ee> написал(а):
>
> On Thu, Aug 31, 2017 at 9:02 AM, Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>> When we have accumulated diff blocknumbers for most of segments we can significantly speed up method of WAL scanning. If we have blocknumbers for all segments we can skip WAL scanning at all.
>
> Have you measured that the WAL scanning is actually a significant
> issue? As a quick experiment I hacked up pg_waldump to just dump block
> references to stdout in binary format. It scanned 2.8GB of WAL in 3.17
> seconds, outputting 9.3M block refs per second. WAL was generated with
> pgbench, synchronous commit off, using 4 cores for 10 minutes - making
> the ratio of work from generating WAL to parsing it be about 750:1.
>

No, I had not done this measurement myself. Sure, parsing WAL, when it is in RAM, is not very expensive. Though, it can be even cheaper before formatting WAL.
I just want to figure out what is the best place for this, if backuping exec is sharing CPUs with postmaster.

Best regards, Andrey Borodin.