RE: Adding REPACK [concurrently]

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, Antonin Houska <ah(at)cybertec(dot)at>, Srinath Reddy Sadipiralla <srinath2133(at)gmail(dot)com>, Mihail Nikalayeu <mihailnikalayeu(at)gmail(dot)com>, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, Pg Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Robert Treat <rob(at)xzilla(dot)net>
Subject: RE: Adding REPACK [concurrently]
Date: 2026-04-10 10:53:53
Message-ID: TYRPR01MB14195633567DA00ABD42570B794592@TYRPR01MB14195.jpnprd01.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

When testing REPACK concurrently, I noticed that all WALs are retained from
the moment REPACK begins copying data to the new table until the command
finishes replaying concurrent changes on the new table and stops the repack
decoding worker.

I understand the reason: the REPACK command itself starts a long-running
transaction, and logical decoding does not advance restart_lsn beyond the
oldest running transaction's start position. As a result, slot.restart_lsn
remains unchanged, preventing the checkpointer from recycling WALs.

However, since REPACK can run for a long time (hours or even days), I'd like
to confirm whether this is expected behavior or if we plan to improve it
in the future ? And additionally, IIUC, REPACK without using concurrent option
does not have this issue.

Given that we do not restart a REPACK, I think the repack decoding worker
should be able to advance restart_lsn each time after writing changes
(similar to how a physical slot behaves). To illustrate this, I've written
a patch (attached) that implements this approach, and it works fine for me.

BTW, catalog_xmin also won't advance, but that seems not a big issue as
the REPACK transaction itself also holds a snapshot that retains catalog tuples,
so advancing catalog_xmin wouldn't change the situation anyway.

Thoughts ?

Best Regards,
Hou zj

Attachment Content-Type Size
v1-0001-Allow-old-WALs-to-be-removed-during-REPACK-CONCUR.patch application/octet-stream 2.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Justin Pryzby 2026-04-10 10:54:42 pg17: XX000: no relation entry for relid 0
Previous Message Jim Jones 2026-04-10 10:28:59 Re: Fix bug with accessing to temporary tables of other sessions