From: | Robert Haas <robertmhaas(at)gmail(dot)com> |
---|---|
To: | Bruce Momjian <bruce(at)momjian(dot)us> |
Cc: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: finding changed blocks using WAL scanning |
Date: | 2019-04-16 01:04:13 |
Message-ID: | CA+TgmoYV3Z7qwZgyO-vMvTiemRFx9Nk-n_kiX5BqBFYAkRn6tw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Mon, Apr 15, 2019 at 4:31 PM Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Can I throw out a simple idea? What if, when we finish writing a WAL
> file, we create a new file 000000010000000000000001.modblock which
> lists all the heap/index files and block numbers modified in that WAL
> file? How much does that help with the list I posted earlier?
>
> I think there is some interesting complexity brought up in this thread.
> Which options are going to minimize storage I/O, network I/O, have only
> background overhead, allow parallel operation, integrate with
> pg_basebackup. Eventually we will need to evaluate the incremental
> backup options against these criteria.
>
> I am thinking tools could retain modblock files along with WAL, could
> pull full-page-writes from WAL, or from PGDATA. It avoids the need to
> scan 16MB WAL files, and the WAL files and modblock files could be
> expired independently.
That is pretty much exactly what I was intending to propose.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2019-04-16 01:07:16 | Re: New vacuum option to do only freezing |
Previous Message | Robert Haas | 2019-04-16 01:03:03 | Re: New vacuum option to do only freezing |