Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

From: Anastasia Lubennikova <a(dot)lubennikova(at)postgrespro(dot)ru>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist
Date: 2018-02-28 16:03:11
Message-ID: a055f129-518e-3eac-1242-833d58edfddb@postgrespro.ru
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I want to propose a bunch of patches which allow to reduce WAL traffic
generated by CREATE INDEX for GiST, GIN and SP-GiST. Similarly to b-tree
and RUM, we can now log index pages of other access methods only once
in the end of indexbuild process. Implementation is based on generic_xlog.

Not only it decreases the amount of WAL generated, but also completely
eliminates WAL overhead in case of error during index build.

I also attached sql scripts which I used to measure xlog size.
They show that pg_wal_lsn_diff for patched version is from 3 to 5 times
smaller.

Not sure if regression tests are needed, since it is just an optimization.
But I do not mind to add them if someone feels that it is necessary.

--
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachment Content-Type Size
0001_generate_xlog_for_rel_v0.patch text/x-patch 1.8 KB
0002_gin_optimal_wal_v0.patch text/x-patch 7.5 KB
0003_gist_optimal_wal_v0.patch text/x-patch 8.7 KB
0004_spgist_optimal_wal_v0.patch text/x-patch 3.3 KB
gin-WAL-test.sql application/sql 714 bytes
gist-WAL-test.sql application/sql 654 bytes
spgist-WAL-test.sql application/sql 673 bytes

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2018-02-28 16:03:57 Re: ON CONFLICT DO UPDATE for partitioned tables
Previous Message Antonin Houska 2018-02-28 16:02:43 Re: [HACKERS] WIP: Aggregation push-down