Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Antonin Houska <ah(at)cybertec(dot)at>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)
Date: 2026-11-20 13:15:58
Message-ID: 7ebfe2c0-a1cc-4561-ac02-cc48f846f1bc@dunslane.net
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Thanks everyone for looking.

On 2026-11-20 Fr 2:54 AM, Antonin Houska wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>
>> In [1], REPACK (CONCURRENTLY) was restricted to heap because nothing in
>> TableAmRoutine says whether an AM can be logically decoded, and without
>> decoding the changes made during the rewrite are silently lost. Masahiko
>> suggested a callback for HEAD, and Bharath that it wait for a field report.
>> Here is one: a table AM we are developing has its own resource manager
>> with rm_decode, and the heap-only check refuses it.
>>
>> The attached patch adds an optional relation_supports_logical_decoding
>> callback, last in TableAmRoutine.  An AM without it is taken not to support
>> decoding, so existing AMs are unaffected.  heap provides it, and REPACK asks
>> the AM instead of comparing its routine with heap's.
> Have you considered adding a callback / field that provides the AM's RmgrId ?
> The AM needs to know it anyway, at least when inserting WAL records (Well, the
> heap AM actually uses two ids, RM_HEAP_ID and RM_HEAP2_ID ...). With this
> knowledge, REPACK (CONCURRENTLY) could check whether the rm_decode() callback
> exists.

I did think about exposing the RmgrId, and it's appealing because
the AM has to know it anyway.  I ended up not going that way, for two
reasons.

First, an AM doesn't have a single rmgr.  Heap uses two, RM_HEAP_ID and
RM_HEAP2_ID, as you say, so we'd need the AM to hand back a list, or
REPACK would have to know which of them matters.  That's more API than a
yes/no question warrants.

Second, a non-NULL rm_decode tells us the rmgr decodes some of its records.
It doesn't tell us that the AM's changes reach the reorder buffer in a form
REPACK can apply, or that every record type the AM writes during a rewrite
is covered.  The AM is the only one in a position to say that, and the
callback lets it.

Do you see a case where the callback alone isn't enough?

On 2026-11-19 Th 9:40 PM, Chao Li wrote:
> After reading the patch, I just have a question: Why does relation_supports_logical_decoding() need a Relation argument? From my understanding, if or not an AM supports logical decoding seems to be an AM-level capability, and the heap AM's implementation only returns true. Is there a known use case where a table AM needs to make this decision based on an individual relation?

I don't have a concrete use case for deciding per relation.  I kept the
Relation argument so that an AM could decide per table, say from a reloption
or a storage mode, without having to change the callback's signature later.

On 2026-11-20 Fr 4:09 AM, Xuneng Zhou wrote:
> The REPACK(CONCURRENTLY) statements failed the wal-level checking with
> wal_level=mininal in my testing. This seems to be the cause of
> 4b2aa4b39cb to move concurrent REPACK tests out of cluster.sql. Does
> it make sense to follow pg_walinspect and set wal_level = replica in a
> temp config for make and meson?

Yes, good catch.  I only ran the test module with the default
wal_level, so I never saw the failure.  With wal_level = minimal, all three
REPACK (CONCURRENTLY) statements fail the wal_level check in
check_concurrent_repack_requirements() before they reach the AM check,
so the
test output is wrong.  v2 follows the pg_walinspect pattern.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Attachment Content-Type Size
v2-0001-Add-a-table-AM-callback-for-logical-decoding-supp.patch text/x-patch 17.5 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Andreas 2026-11-20 13:36:09 Re: O(N^3) planning time in choose_plan_name() since 8c49a484e8e
Previous Message Tomas Vondra 2026-11-20 13:11:52 Re: Parallel GIN, BRIN builds fail to maintain ii_BrokenHotChain