Re: Disable WAL logging to speed up data loading

From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
Cc: osumi(dot)takamichi(at)fujitsu(dot)com, masao(dot)fujii(at)oss(dot)nttdata(dot)com, ashutosh(dot)bapat(dot)oss(at)gmail(dot)com, tsunakawa(dot)takay(at)fujitsu(dot)com, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Disable WAL logging to speed up data loading
Date: 2020-11-10 14:33:12
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


* Kyotaro Horiguchi (horikyota(dot)ntt(at)gmail(dot)com) wrote:
> For fuel(?) of the discussion, I tried a very-quick PoC for in-place
> ALTER TABLE SET LOGGED/UNLOGGED and resulted as attached. After some
> trials of several ways, I drifted to the following way after poking
> several ways.
> 1. Flip BM_PERMANENT of active buffers
> 2. adding/removing init fork
> 3. sync files,
> 4. Flip pg_class.relpersistence.
> It always skips table copy in the SET UNLOGGED case, and only when
> wal_level=minimal in the SET LOGGED case. Crash recovery seems
> working by some brief testing by hand.

Somehow missed that this patch more-or-less does what I was referring to
down-thread, but I did want to mention that it looks like it's missing a
necessary FlushRelationBuffers() call before the sync, otherwise there
could be dirty buffers for the relation that's being set to LOGGED (with
wal_level=minimal), which wouldn't be good. See the comments above

> Of course, I haven't performed intensive test on it.

Reading through the thread, it didn't seem very clear, but we should
definitely make sure that it does the right thing on replicas when going
between unlogged and logged (and between logged and unlogged too), of



In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Georgios Kokolatos 2020-11-10 15:03:28 Re: Add session statistics to pg_stat_database
Previous Message gkokolatos 2020-11-10 14:19:06 Re: PATCH: Attempt to make dbsize a bit more consistent