Re: RFC: logical publication via inheritance root?

From: Jacob Champion <jchampion(at)timescale(dot)com>
To: Aleksander Alekseev <aleksander(at)timescale(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Peter Smith <smithpb2250(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Subject: Re: RFC: logical publication via inheritance root?
Date: 2023-08-31 18:16:49
Message-ID: CAAWbhmjcnoV7Xu6LHr_hxqWmVtehv404bvDye+QZcUDSg8NSKw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 7/19/23 04:54, Aleksander Alekseev wrote:
> I decided to take a closer look at 0001.

Hi Aleks! I saw you put this back into Needs Review; thanks.

This thread has been pretty quiet from me, because we've run into
difficulties on the subscriber end. Our original optimistic assumption
was that we just needed to funnel all the leaf tables through the root
on the publisher, and then a sufficiently complex replica trigger on
the subscriber would be able to correctly route the data through the
root into new leaves.

That has not panned out for a number of reasons. Probably the easiest
one to describe is that replica identity handling breaks: if we move
the incoming tuples to different tables, and an UPDATE or DELETE comes
in later for those rows, the replication logic checks the root table
(bypassing our existing routing logic) and sees that they don't exist.
We never get the chance to handle routing the way that partitions do
[1]. Given that, I think I need to pivot and focus on the subscriber
side first. That might(?) be a smaller effort anyway, and if we can't
make headway there then publisher-side support probably doesn't make
sense at all.

So I'll pause this CF entry for now. This would also be a good time to
ask the crowd: are there alternative approaches to solve the OP that I
may be missing?

Thanks!
--Jacob

[1] https://git.postgresql.org/cgit/postgresql.git/tree/src/backend/replication/logical/worker.c?h=8bf7db02#n2631

P.S. I've attached a v4, which fixes the semantics problem I mentioned
upthread, so it doesn't get lost in the shuffle.

Attachment Content-Type Size
since-v3.diff.txt text/plain 36.8 KB
v4-0001-pgoutput-refactor-publication-cache-construction.patch text/x-patch 18.1 KB
v4-0002-WIP-introduce-publish_via_parent-for-logical-repl.patch text/x-patch 87.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Dagfinn Ilmari Mannsåker 2023-08-31 18:28:38 Re: Replace some cstring_to_text to cstring_to_text_with_len
Previous Message Muhammad Malik 2023-08-31 18:12:34 Re: Improve heapgetpage() performance, overhead from serializable