Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

From: Andrey Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>
To: Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, a(dot)lubennikova(at)postgrespro(dot)ru, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist
Date: 2019-02-06 10:08:56
Message-ID: ce454ce6-e970-7bf3-827f-32a8e406273d@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

The patchset had a problem with all-zero pages, has appeared at index
build stage: the generic_log_relation() routine sends all pages into the
WAL. So lsn field at all-zero page was initialized and the
PageIsVerified() routine detects it as a bad page.
The solution may be:
1. To improve index build algorithms and eliminate the possibility of
not used pages appearing.
2. To mark each page as 'dirty' right after initialization. In this case
we will got 'empty' page instead of the all-zeroed.
3. Do not write into the WAL all-zero pages.

In the patchset (see attachment) I used approach No.3.

On 04.02.2019 10:04, Michael Paquier wrote:
> On Tue, Dec 18, 2018 at 10:41:48AM +0500, Andrey Lepikhov wrote:
>> Ok. It is used only for demonstration.
>
> The latest patch set needs a rebase, so moved to next CF, waiting on
> author as this got no reviews.
> --
> Michael
>

--
Andrey Lepikhov
Postgres Professional
https://postgrespro.com
The Russian Postgres Company

Attachment Content-Type Size
0001-Relation-into-WAL-function.patch text/x-patch 2.3 KB
0002-GIN-Optimal-WAL-Usage.patch text/x-patch 10.3 KB
0003-GiST-Optimal-WAL-Usage.patch text/x-patch 11.0 KB
0004-SP-GiST-Optimal-WAL-Usage.patch text/x-patch 6.3 KB
pEpkey.asc application/pgp-keys 2.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2019-02-06 10:14:38 Re: Add pg_partition_root to get top-most parent of a partition tree
Previous Message Amit Langote 2019-02-06 09:59:27 Re: Bogus lateral-reference-propagation logic in create_lateral_join_info