Re: [HACKERS] WAL logging problem in 9.4.3?

Hello. I revised the patch I think addressing all your comments.

Differences from v7 patch are:

v9-0001:

- Renamed the script from 016_ to 017_.

- Added some additional tests.

v9-0002:
- Fixed _bt_blwritepage().
It is re-modified by v9-0007.

v9-0003: New patch.
- Refactors out xlog sutff from heap_insert/delete.
(log_heap_insert(), log_heap_udpate())

v9-0004: (v7-0003, v8-0004)
- Renamed some struct names and member names.
(PendingRelSync -> RelWalRequirement
.sync_above -> skip_wal_min_blk, .truncated_to -> wal_log_min_blk)

- Rename the addtional members in RelationData to rd_*.

- Explicitly initialize the additional members only in
load_relcache_init_file().

- Added new interface functions that accept block number and
SMgrRelation.
(BlockNeedsWAL(), RecordPendingSync())

- Support subtransaction, (or invalidation).
(RelWalRequirement.create_sxid, invalidate_sxid,
RelationInvalidateWALRequirements(), smgrDoPendingSyncs())

- Support forks.
(RelWalRequirement.forks, smgrDoPendingSyncs(), RecordPendingSync())

- Removd elog(LOG)s and a leftover comment.

v9-0005: (v7-0004, v8-0005)

- Fixed heap_update().
(heap_update())

v9-0006: New patch.

- Modifies CLUSTER to skip WAL logging.

v9-0007: New patch.

- Modifies ALTER TABLE SET TABLESPACE to skip WAL logging.

v9-0008: New patch.

- Modifies btbuild to skip WAL logging.

- Modifies btinsertonpg to skip WAL logging after truncation.

- Overrites on v9-0002's change.

ALL:

- Rebased.

- Fixed typos and mistakes in comments.

> At Wed, 20 Mar 2019 17:17:54 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote in <20190320(dot)171754(dot)171896368(dot)horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
> > > We still use heap_sync() in CLUSTER. Can we migrate CLUSTER to the newer
> > > heap_register_sync()? Patch v7 makes some commands use the new way (COPY,
> > > CREATE TABLE AS, REFRESH MATERIALIZED VIEW, ALTER TABLE) and leaves other
> > > commands using the old way (CREATE INDEX USING btree, ALTER TABLE SET
> > > TABLESPACE, CLUSTER). It would make the system simpler to understand if we
> > > eliminated the old way. If that creates more problems than it solves, please
> > > at least write down a coding rule to explain why certain commands shouldn't
> > > use the old way.
> >
> > Perhaps doable for TABLESPACE and CLUSTER. I'm not sure about
> > CREATE INDEX. I'll consider them.
>
> I added the CLUSTER case in the new patchset. For the SET
> TABLESPACE case, it works on SMGR layer and manipulates fork
> files explicitly but this stuff is Relation based and doesn't
> distinguish forks. We can modify this stuff to work on smgr and
> make it fork-aware but I don't think it is worth doing.
>
> CREATE INDEX is not changed in this version. I continue to
> consider it.

I managed to simplify the change. Please look at v9-0008.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center