| From: | Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> |
|---|---|
| To: | Srinath Reddy Sadipiralla <srinath2133(at)gmail(dot)com> |
| Cc: | Mihail Nikalayeu <mihailnikalayeu(at)gmail(dot)com>, Antonin Houska <ah(at)cybertec(dot)at>, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, Pg Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Robert Treat <rob(at)xzilla(dot)net> |
| Subject: | Re: Adding REPACK [concurrently] |
| Date: | 2026-03-25 20:12:48 |
| Message-ID: | 202603252005.quy5h4oipoxd@alvherre.pgsql |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On 2026-Mar-25, Srinath Reddy Sadipiralla wrote:
> Then as suggested by Alvaro off-list I checked the lock upgrade
> behavior during the table swap phase. I observed that if another
> transaction holds a conflicting lock on the table when the swap is
> attempted, it can lead to “transient table” data loss during a manual
> or timeout abort. when a REPACK (concurrent) waits for a conflicting
> lock to be released and eventually hits a lock_timeout (or is
> cancelled via ctrl+c), the transaction aborts. During this abort, the
> cleanup process triggers smgrDoPendingDeletes. This results in the
> removal of all transient table relfiles and decoder worker files
> created during the process. This effectively wipes out the work done
> by the transient table creation before the swap could successfully
> complete, this happens because during transient table creation we add
> the table to the PendingRelDelete list.
I think we certainly need to make the files be deleted in some
reasonable fashion if repack fails partway through.
As for lock upgrade, I wonder if the best way to handle this isn't to
hack the deadlock detector so that it causes any *other* process to die,
if they detect that they would block on REPACK. Arguably there's
nothing that you can do to a table while its undergoing REPACK
CONCURRENTLY; any alterations would have to wait until the repacking is
compelted. We can implement that idea simply enough, as shown in this
crude prototype. (I omitted the last three patches in the series, and
squashed my proposed changes into 0003, as announced in my previous
posting.)
The isolation test file is also a bit crude; I just copied repack.spec
to a new file and removed the uninteresting bits.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"I am amazed at [the pgsql-sql] mailing list for the wonderful support, and
lack of hesitasion in answering a lost soul's question, I just wished the rest
of the mailing list could be like this." (Fotis)
https://postgr.es/m/200606261359.k5QDxE2p004593@auth-smtp.hol.gr
| Attachment | Content-Type | Size |
|---|---|---|
| v44b-0001-Make-index_concurrently_create_copy-more-genera.patch | text/x-diff | 9.4 KB |
| v44b-0002-Do-not-dereference-varattrib_4b-in-VARSIZE_4B.patch | text/x-diff | 1.9 KB |
| v44b-0003-Add-CONCURRENTLY-option-to-REPACK-command.patch | text/x-diff | 180.8 KB |
| v44b-0004-hack-deadlock-detector.patch | text/x-diff | 8.0 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tomas Vondra | 2026-03-25 20:23:54 | Re: Proposal: Adding compression of temporary files |
| Previous Message | Bertrand Drouvot | 2026-03-25 19:54:42 | Re: Adding locks statistics |