Re: POC: Cleaning up orphaned files using undo logs

From: Dmitry Dolgov <9erthalion6(at)gmail(dot)com>
To: Antonin Houska <ah(at)cybertec(dot)at>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: POC: Cleaning up orphaned files using undo logs
Date: 2021-09-21 08:07:55
Message-ID: CA+q6zcUMgY=P56W3FeHohrS0A+0jzcoJQ0w9sS2-5F0_F2WdHw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, 21 Sep 2021 09:00 Antonin Houska, <ah(at)cybertec(dot)at> wrote:

> Dmitry Dolgov <9erthalion6(at)gmail(dot)com> wrote:
>
> > Yep, makes sense, thanks. I have few more questions:
> >
> > * The use case with orphaned files is working somewhat differently after
> > the rebase on the latest master, do you observe it as well? The
> > difference is ApplyPendingUndo -> SyncPostCheckpoint doesn't clean up
> > an orphaned relation file immediately (only later on checkpoint)
> > because of empty pendingUnlinks. I haven't investigated more yet, but
> > seems like after this commit:
> >
> > commit 7ff23c6d277d1d90478a51f0dd81414d343f3850
> > Author: Thomas Munro <tmunro(at)postgresql(dot)org>
> > Date: Mon Aug 2 17:32:20 2021 +1200
> >
> > Run checkpointer and bgwriter in crash recovery.
> >
> > Start up the checkpointer and bgwriter during crash recovery
> (except in
> > --single mode), as we do for replication. This wasn't done back
> in
> > commit cdd46c76 out of caution. Now it seems like a better idea
> to make
> > the environment as similar as possible in both cases. There may
> also be
> > some performance advantages.
> >
> > something has to be updated (pendingOps are empty right now, so no
> > unlink request is remembered).
>
> I haven't been debugging that part recently, but yes, this commit is
> relevant,
> thanks for pointing that out! Attached is a patch that should fix it. I'll
> include it in the next version of the patch series, unless you tell me that
> something is still wrong.
>

Sure, but I can take a look only in a couple of days.

>

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Fabrice Chapuis 2021-09-21 08:22:32 Re: Logical replication timeout problem
Previous Message Antonin Houska 2021-09-21 07:02:02 Re: POC: Cleaning up orphaned files using undo logs