Re: Adding REPACK [concurrently]

From: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
To: Srinath Reddy Sadipiralla <srinath2133(at)gmail(dot)com>
Cc: Mihail Nikalayeu <mihailnikalayeu(at)gmail(dot)com>, Antonin Houska <ah(at)cybertec(dot)at>, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, Pg Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Robert Treat <rob(at)xzilla(dot)net>
Subject: Re: Adding REPACK [concurrently]
Date: 2026-03-25 20:12:48
Message-ID: 202603252005.quy5h4oipoxd@alvherre.pgsql
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2026-Mar-25, Srinath Reddy Sadipiralla wrote:

> Then as suggested by Alvaro off-list I checked the lock upgrade
> behavior during the table swap phase. I observed that if another
> transaction holds a conflicting lock on the table when the swap is
> attempted, it can lead to “transient table” data loss during a manual
> or timeout abort. when a REPACK (concurrent) waits for a conflicting
> lock to be released and eventually hits a lock_timeout (or is
> cancelled via ctrl+c), the transaction aborts. During this abort, the
> cleanup process triggers smgrDoPendingDeletes. This results in the
> removal of all transient table relfiles and decoder worker files
> created during the process. This effectively wipes out the work done
> by the transient table creation before the swap could successfully
> complete, this happens because during transient table creation we add
> the table to the PendingRelDelete list.

I think we certainly need to make the files be deleted in some
reasonable fashion if repack fails partway through.

As for lock upgrade, I wonder if the best way to handle this isn't to
hack the deadlock detector so that it causes any *other* process to die,
if they detect that they would block on REPACK. Arguably there's
nothing that you can do to a table while its undergoing REPACK
CONCURRENTLY; any alterations would have to wait until the repacking is
compelted. We can implement that idea simply enough, as shown in this
crude prototype. (I omitted the last three patches in the series, and
squashed my proposed changes into 0003, as announced in my previous
posting.)

The isolation test file is also a bit crude; I just copied repack.spec
to a new file and removed the uninteresting bits.

--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"I am amazed at [the pgsql-sql] mailing list for the wonderful support, and
lack of hesitasion in answering a lost soul's question, I just wished the rest
of the mailing list could be like this." (Fotis)
https://postgr.es/m/200606261359.k5QDxE2p004593@auth-smtp.hol.gr

Attachment Content-Type Size
v44b-0001-Make-index_concurrently_create_copy-more-genera.patch text/x-diff 9.4 KB
v44b-0002-Do-not-dereference-varattrib_4b-in-VARSIZE_4B.patch text/x-diff 1.9 KB
v44b-0003-Add-CONCURRENTLY-option-to-REPACK-command.patch text/x-diff 180.8 KB
v44b-0004-hack-deadlock-detector.patch text/x-diff 8.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2026-03-25 20:23:54 Re: Proposal: Adding compression of temporary files
Previous Message Bertrand Drouvot 2026-03-25 19:54:42 Re: Adding locks statistics