| From: | Andrew Dunstan <andrew(at)dunslane(dot)net> |
|---|---|
| To: | Antonin Houska <ah(at)cybertec(dot)at> |
| Cc: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY) |
| Date: | 2026-11-20 13:15:58 |
| Message-ID: | 7ebfe2c0-a1cc-4561-ac02-cc48f846f1bc@dunslane.net |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Thanks everyone for looking.
On 2026-11-20 Fr 2:54 AM, Antonin Houska wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>
>> In [1], REPACK (CONCURRENTLY) was restricted to heap because nothing in
>> TableAmRoutine says whether an AM can be logically decoded, and without
>> decoding the changes made during the rewrite are silently lost. Masahiko
>> suggested a callback for HEAD, and Bharath that it wait for a field report.
>> Here is one: a table AM we are developing has its own resource manager
>> with rm_decode, and the heap-only check refuses it.
>>
>> The attached patch adds an optional relation_supports_logical_decoding
>> callback, last in TableAmRoutine. An AM without it is taken not to support
>> decoding, so existing AMs are unaffected. heap provides it, and REPACK asks
>> the AM instead of comparing its routine with heap's.
> Have you considered adding a callback / field that provides the AM's RmgrId ?
> The AM needs to know it anyway, at least when inserting WAL records (Well, the
> heap AM actually uses two ids, RM_HEAP_ID and RM_HEAP2_ID ...). With this
> knowledge, REPACK (CONCURRENTLY) could check whether the rm_decode() callback
> exists.
I did think about exposing the RmgrId, and it's appealing because
the AM has to know it anyway. I ended up not going that way, for two
reasons.
First, an AM doesn't have a single rmgr. Heap uses two, RM_HEAP_ID and
RM_HEAP2_ID, as you say, so we'd need the AM to hand back a list, or
REPACK would have to know which of them matters. That's more API than a
yes/no question warrants.
Second, a non-NULL rm_decode tells us the rmgr decodes some of its records.
It doesn't tell us that the AM's changes reach the reorder buffer in a form
REPACK can apply, or that every record type the AM writes during a rewrite
is covered. The AM is the only one in a position to say that, and the
callback lets it.
Do you see a case where the callback alone isn't enough?
On 2026-11-19 Th 9:40 PM, Chao Li wrote:
> After reading the patch, I just have a question: Why does relation_supports_logical_decoding() need a Relation argument? From my understanding, if or not an AM supports logical decoding seems to be an AM-level capability, and the heap AM's implementation only returns true. Is there a known use case where a table AM needs to make this decision based on an individual relation?
I don't have a concrete use case for deciding per relation. I kept the
Relation argument so that an AM could decide per table, say from a reloption
or a storage mode, without having to change the callback's signature later.
On 2026-11-20 Fr 4:09 AM, Xuneng Zhou wrote:
> The REPACK(CONCURRENTLY) statements failed the wal-level checking with
> wal_level=mininal in my testing. This seems to be the cause of
> 4b2aa4b39cb to move concurrent REPACK tests out of cluster.sql. Does
> it make sense to follow pg_walinspect and set wal_level = replica in a
> temp config for make and meson?
Yes, good catch. I only ran the test module with the default
wal_level, so I never saw the failure. With wal_level = minimal, all three
REPACK (CONCURRENTLY) statements fail the wal_level check in
check_concurrent_repack_requirements() before they reach the AM check,
so the
test output is wrong. v2 follows the pg_walinspect pattern.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
| Attachment | Content-Type | Size |
|---|---|---|
| v2-0001-Add-a-table-AM-callback-for-logical-decoding-supp.patch | text/x-patch | 17.5 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Andreas | 2026-11-20 13:36:09 | Re: O(N^3) planning time in choose_plan_name() since 8c49a484e8e |
| Previous Message | Tomas Vondra | 2026-11-20 13:11:52 | Re: Parallel GIN, BRIN builds fail to maintain ii_BrokenHotChain |