Re: Logical insert/update/delete WAL records for custom table AMs

On Wed, Mar 30, 2022 at 2:31 PM Jeff Davis <pgsql(at)j-davis(dot)com> wrote:
> Attached a new version. The scope expanded, so this is likely to slip
> v15 at this late time. For 15, I'll focus on my extensible rmgr work,
> which can serve similar purposes.
>
> The main purpose of this patch is to be able to emit logical events for
> a table (insert/update/delete/truncate) without actually modifying a
> table or relying on the heap at all. That allows table AMs to support
> logical decoding/replication, and perhaps allows a few other
> interesting use cases (maybe foreign tables?). There are really two
> advantages of this approach over relying on a custom rmgr:
>
> 1. Easier for extension authors
> 2. Doesn't require an extension module to be loaded to start the
> server

I'm not sure that I understand exactly how this is intended to be
used. I can think of three possibilities:

1. An AM that doesn't care about having anything happening during
recovery, but wants to be able to get logical decoding to do some
work. Maybe the intention of the AM is that data is available only
when the server is not in recovery and all data is lost on shutdown,
or maybe the AM has its own separate durability mechanism.

2. An AM that wants things to happen during recovery, but handles that
separately. For example, maybe it logs all the data changes via
log_newpage() and then separately emits these log records.

3. An AM that wants things to happen during recovery, and expects
these records to serve that purpose. That would require getting
control during WAL replay as well as during decoding, and would
probably only work for an AM whose data is not block-structured (e.g.
an in-memory btree that is dumped to disk at every checkpoint).

My guess is that this is intended to meet use cases 1 and 2 but not 3.
Is that correct?

--
Robert Haas
EDB: http://www.enterprisedb.com