Re: POC: Cleaning up orphaned files using undo logs

From: Antonin Houska <ah(at)cybertec(dot)at>
To: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Cc: Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: POC: Cleaning up orphaned files using undo logs
Date: 2021-09-29 06:09:34
Message-ID: 54406.1632895774@antos
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:

> On Wed, Sep 29, 2021 at 8:18 AM Antonin Houska <ah(at)cybertec(dot)at> wrote:
> > I'm just trying to use the existing infrastructure: the effect of DROP TABLE
> > also appear to be performed by the checkpointer. However I don't know why the
> > unlinks need to be performed by the checkpointer.
>
> For DROP TABLE, we leave an empty file (I've been calling it a
> "tombstone file") so that GetNewRelFileNode() won't let you reuse the
> same relfilenode in the same checkpoint cycle. One reason is that
> wal_level=minimal has a data-eating crash recovery failure mode if you
> reuse a relfilenode in a checkpoint cycle.

Interesting. Is the problem that REDO of the DROP TABLE command deletes the
relfilenode which already contains the new data, but the new data cannot be
recovered because (due to wal_level=minimal) it's not present in WAL? In this
case I suppose that the checkpoint just ensures that the DROP TABLE won't be
replayed during the next crash recovery.

BTW, does that comment fix attached make sense to you? The corresponding code
in InitSync() is

/*
* Create pending-operations hashtable if we need it. Currently, we need
* it if we are standalone (not under a postmaster) or if we are a
* checkpointer auxiliary process.
*/
if (!IsUnderPostmaster || AmCheckpointerProcess())

I suspect this is also related to the commit 7ff23c6d27.

Thanks for your answer, I was considering to add you to CC :-)

--
Antonin Houska
Web: https://www.cybertec-postgresql.com

Attachment Content-Type Size
sync_comment.patch text/x-diff 505 bytes

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Greg Nancarrow 2021-09-29 06:19:06 Re: Added schema level support for publication.
Previous Message osumi.takamichi@fujitsu.com 2021-09-29 06:05:45 RE: Failed transaction statistics to measure the logical replication progress