Re: [HACKERS] WAL logging problem in 9.4.3?

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: noah(at)leadboat(dot)com
Cc: 9erthalion6(at)gmail(dot)com, andrew(dot)dunstan(at)2ndquadrant(dot)com, hlinnaka(at)iki(dot)fi, robertmhaas(at)gmail(dot)com, michael(at)paquier(dot)xyz, pgsql-hackers(at)postgresql(dot)org
Subject: Re: [HACKERS] WAL logging problem in 9.4.3?
Date: 2019-03-26 07:35:07
Message-ID: 20190326.163507.239339952.horiguchi.kyotaro@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

Hello. I revised the patch I think addressing all your comments.

Differences from v7 patch are:

v9-0001:

- Renamed the script from 016_ to 017_.

- Added some additional tests.

v9-0002:
- Fixed _bt_blwritepage().
It is re-modified by v9-0007.

v9-0003: New patch.
- Refactors out xlog sutff from heap_insert/delete.
(log_heap_insert(), log_heap_udpate())

v9-0004: (v7-0003, v8-0004)
- Renamed some struct names and member names.
(PendingRelSync -> RelWalRequirement
.sync_above -> skip_wal_min_blk, .truncated_to -> wal_log_min_blk)

- Rename the addtional members in RelationData to rd_*.

- Explicitly initialize the additional members only in
load_relcache_init_file().

- Added new interface functions that accept block number and
SMgrRelation.
(BlockNeedsWAL(), RecordPendingSync())

- Support subtransaction, (or invalidation).
(RelWalRequirement.create_sxid, invalidate_sxid,
RelationInvalidateWALRequirements(), smgrDoPendingSyncs())

- Support forks.
(RelWalRequirement.forks, smgrDoPendingSyncs(), RecordPendingSync())

- Removd elog(LOG)s and a leftover comment.

v9-0005: (v7-0004, v8-0005)

- Fixed heap_update().
(heap_update())

v9-0006: New patch.

- Modifies CLUSTER to skip WAL logging.

v9-0007: New patch.

- Modifies ALTER TABLE SET TABLESPACE to skip WAL logging.

v9-0008: New patch.

- Modifies btbuild to skip WAL logging.

- Modifies btinsertonpg to skip WAL logging after truncation.

- Overrites on v9-0002's change.

ALL:

- Rebased.

- Fixed typos and mistakes in comments.

> At Wed, 20 Mar 2019 17:17:54 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote in <20190320(dot)171754(dot)171896368(dot)horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
> > > We still use heap_sync() in CLUSTER. Can we migrate CLUSTER to the newer
> > > heap_register_sync()? Patch v7 makes some commands use the new way (COPY,
> > > CREATE TABLE AS, REFRESH MATERIALIZED VIEW, ALTER TABLE) and leaves other
> > > commands using the old way (CREATE INDEX USING btree, ALTER TABLE SET
> > > TABLESPACE, CLUSTER). It would make the system simpler to understand if we
> > > eliminated the old way. If that creates more problems than it solves, please
> > > at least write down a coding rule to explain why certain commands shouldn't
> > > use the old way.
> >
> > Perhaps doable for TABLESPACE and CLUSTER. I'm not sure about
> > CREATE INDEX. I'll consider them.
>
> I added the CLUSTER case in the new patchset. For the SET
> TABLESPACE case, it works on SMGR layer and manipulates fork
> files explicitly but this stuff is Relation based and doesn't
> distinguish forks. We can modify this stuff to work on smgr and
> make it fork-aware but I don't think it is worth doing.
>
> CREATE INDEX is not changed in this version. I continue to
> consider it.

I managed to simplify the change. Please look at v9-0008.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
v9-0001-TAP-test-for-copy-truncation-optimization.patch text/x-patch 9.5 KB
v9-0002-Write-WAL-for-empty-nbtree-index-build.patch text/x-patch 1.6 KB
v9-0003-Move-XLOG-stuff-from-heap_insert-and-heap_delete.patch text/x-patch 11.5 KB
v9-0004-Add-infrastructure-to-WAL-logging-skip-feature.patch text/x-patch 25.4 KB
v9-0005-Fix-WAL-skipping-feature.patch text/x-patch 19.0 KB
v9-0006-Change-cluster-to-use-the-new-pending-sync-infrastru.patch text/x-patch 6.1 KB
v9-0007-Change-ALTER-TABLESPACE-to-use-the-pending-sync-infr.patch text/x-patch 3.7 KB
v9-0008-Optimize-WAL-logging-on-btree-bulk-insertion.patch text/x-patch 4.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Surafel Temesgen 2019-03-26 07:46:00 Re: Re: FETCH FIRST clause WITH TIES option
Previous Message Amit Langote 2019-03-26 07:16:41 Re: speeding up planning with partitions