Re: Per-table resync for logical replication subscriptions

From: Cagri Biroglu <cagri(dot)biroglu(at)adyen(dot)com>
To: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
Cc: "smithpb2250(at)gmail(dot)com" <smithpb2250(at)gmail(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Subject: Re: Per-table resync for logical replication subscriptions
Date: 2026-09-01 07:17:02
Message-ID: CAA36msodPo-2JHL-G9yn-zU_80sg9OSiTFKoeySafYtDSWRHrA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dear Hayato,

Thank you .

v6 is attached, along with the reproducer for point 04.

> 01. Can you clarify the reason why ExecuteTruncate() is not used here?
> It does truncate_check_rel() and truncate_check_perms(). Are they
> needed here? If not, please add a comment. At least,
> InvokeObjectTruncateHoo() seems needed.

ExecuteTruncate() is not usable because it starts from a list of RangeVars
and re-resolves them, whereas by that point we have already resolved,
validated and locked the relations, and the whole set has to be truncated in
one call so that foreign keys between its members are satisfied.

On the checks themselves, you are right that some of them were missing.
ExecuteTruncateGuts() applies them only to the relations it discovers itself
through CASCADE and leaves the explicitly passed ones to the caller, so as a
direct caller we owe them.

Going through what they do:

truncate_check_rel() rejects non-tables and system catalogs. Neither can
occur for a relation that is in pg_subscription_rel, and sequences are
rejected earlier with a more helpful message.

truncate_check_perms() is the ACL_TRUNCATE check. The first pass already
does this explicitly.

truncate_check_activity() rejects other backends' temp tables and calls
CheckTableNotInUse(). The temp case cannot arise, as a temp table cannot
be
a partition of a permanent one, but CheckTableNotInUse() was missing.

InvokeObjectTruncateHook(), which as you say was missing.

Worth noting for the archives: the hook is not a separate call. It is the
last statement of truncate_check_rel(), so a caller only fires it by going
through that function. apply_handle_truncate() calls none of the three, and
therefore does not fire the hook either. I still think you are right that we
should. That path is replication applying a remote TRUNCATE; ours is a user
typing a command that discards the contents of a table, which is precisely
what an audit or label provider wants to see, and it is also why we already
require TRUNCATE privilege where apply does not.

> 02. Question: this command drops replication origins and remote slots,
> which cannot be rolled back. So what should users do if the command
> fails?

> 03. Such operations should be done after the transactional ones
> (i.e., ExecuteTruncateGuts()).

Taking 03 first, since it changes the answer to 02.

You are right, and the origin drop was worse placed than it looks.
replorigin_drop_by_name() is half transactional: CatalogTupleDelete() rolls
back, but replorigin_state_clear() emits XLOG_REPLORIGIN_DROP and clears the
in-memory progress immediately, and that part does not roll back. So the old
ordering could wipe an origin's progress and then abort in the truncate, on
nothing more exotic than a foreign key reference or a lock timeout, leaving
the origin row restored but its progress gone.

v6 moves the origin drops after ExecuteTruncateGuts() and after the state
resets, so the order is now: everything that can fail for an ordinary
reason,
then the origin drops, then the slot drops last of all as the only step
whose
effect leaves this node.

That gives 02 a fairly narrow answer. Every ordinary failure , name
resolution, subscription membership, privileges, the cross-subscription
check, the truncate, the state resets ; now happens before anything
irreversible, so the command is all-or-nothing and there is nothing for the
user to do but fix the cause and re-run.

What remains is a failure inside the final phase itself, for instance the
publisher becoming unreachable partway through the slot drops. Then the
transaction rolls back, the catalog is unchanged, and some tablesync slots
are already gone. The answer there is simply to re-run the command: the
slots
are dropped with missing_ok, so the ones already gone are skipped and the
command converges. This is the same bargain DROP SUBSCRIPTION makes, and for
the same reason, which is also why both are PreventInTransactionBlock().

> 04. IIUC ATTACH PARTITION and DETACH PARTITION CONCURRENTLY require
> only ShareUpdateExclusiveLock for the root table. This means that
> some tables can be missed from the truncation, or detached tables
> can be truncated. A primitive idea is to check whether the schema
> structure is modified after acquiring the AccessExclusive lock;
> there may be better approaches.

Confirmed, and fixed as you suggest.

Best regards,
Cagri Biroglu

On Fri, Aug 21, 2026 at 5:54 AM Hayato Kuroda (Fujitsu) <
kuroda(dot)hayato(at)fujitsu(dot)com> wrote:

> Dear Cagri,
>
> Hi, thanks for updating the patch. Sorry I could not reply to your latest
> post,
> because it was blocked by the company's rule.
>
> Few comments:
> 01.
> According to ExecuteTruncate(), we did some additional check like
> truncate_check_rel() and truncate_check_perms(). Is it needed here? If not
> we can
> add code comment here. At least, InvokeObjectTruncateHoo() seems needed.
>
> 02. Question:
> This command drops replication origins and remote slots, which cannot be
> rolled back.
> So what should users do the command fails?
>
> 03.
> Also, such operations should be done after the transctional ones
> (ExecuteTruncateGuts).
>
> 04.
> I found that ATTACH PARTITION/DETACH PARTITION CONCURRENTLY command
> acquires the
> ShareUpdateExclusive Lock for the root table. This can cause that tables
> are
> missed to be truncated, or detached tables can be truncated.
> One primitive idea for fixing is to check whether the schema structure is
> modified
> after acquiring the AccesExclusive lock, but there may be better
> approaches.
>
> Best regards,
> Hayato Kuroda
> FUJITSU LIMITED
>
>

Attachment Content-Type Size
v6-0001-refresh-table.patch application/octet-stream 65.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message David Rowley 2026-09-01 07:18:17 Re: Reducing relcache memory usage: deduping index shapes
Previous Message jian he 2026-09-01 07:13:14 Re: COPY FROM with RLS