Re: In-placre persistance change of a relation

From: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
To: Jakub(dot)Wartak(at)tomtom(dot)com
Cc: tsunakawa(dot)takay(at)fujitsu(dot)com, osumi(dot)takamichi(at)fujitsu(dot)com, sfrost(at)snowman(dot)net, masao(dot)fujii(at)oss(dot)nttdata(dot)com, ashutosh(dot)bapat(dot)oss(at)gmail(dot)com, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: In-placre persistance change of a relation
Date: 2021-12-21 11:04:55
Message-ID: 20211221.200455.837892680926943849.horikyota.ntt@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

At Tue, 21 Dec 2021 17:13:21 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> Ugh! I completely forgot about TAP tests.. Thanks for the testing and
> sorry for the bugs.
>
> This is a bit big change so I need a bit of time before posting the
> next version.

I took a bit too long detour but the patch gets to pass make-world for
me.

In this version:

- When relation persistence is changed from logged to unlogged, buffer
persistence is flipped then an init-fork is created along with a mark
file for the fork (RelationCreateInitFork). The mark file is removed
at commit but left alone after a crash before commit. At the next
startup, ResetUnloggedRelationsInDbspaceDir() removes the init fork
file if it finds the mark file corresponding to the file.

- When relation persistence is changed from unlogged to logged, buffer
persistence is flipped then the exisging init-fork is marked to be
dropped at commit (RelationDropInitFork). Finally the whole content
is WAL-logged in the page-wise manner (RelationChangePersistence),

- The two operations above are repeatable within a transaction and
commit makes the last operation persist and rollback make the all
operations abandoned.

- Storage files are created along with a "mark" file for the
relfilenode. It behaves the same way to the above except the mark
files corresponds to the whole relfilenode.

- The at-commit operations this patch adds require to be WAL-logged so
they don't fit pendingDeletes list, which is executed after commit. I
added a new pending-work list pendingCleanups that is executed just
after pendingSyncs. (new in this version)


regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
v10-0001-In-place-table-persistence-change.patch text/x-patch 75.0 KB
v10-0002-New-command-ALTER-TABLE-ALL-IN-TABLESPACE-SET-LO.patch text/x-patch 11.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dilip Kumar 2021-12-21 11:17:23 Checkpointer crashes with "PANIC: could not fsync file "pg_tblspc/.."
Previous Message vignesh C 2021-12-21 10:03:07 Re: row filtering for logical replication