RE: row filtering for logical replication

From: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Peter Smith <smithpb2250(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Euler Taveira <euler(at)eulerto(dot)com>, Greg Nancarrow <gregn4422(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, Ajin Cherian <itsajin(at)gmail(dot)com>, "tanghy(dot)fnst(at)fujitsu(dot)com" <tanghy(dot)fnst(at)fujitsu(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Rahila Syed <rahilasyed90(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, Önder Kalacı <onderkalaci(at)gmail(dot)com>, japin <japinli(at)hotmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, David Steele <david(at)pgmasters(dot)net>, Craig Ringer <craig(at)2ndquadrant(dot)com>, Amit Langote <amitlangote09(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: row filtering for logical replication
Date: 2022-02-01 03:45:21
Message-ID: OS0PR01MB57162EE8FADB2E8EBDEE85E594269@OS0PR01MB5716.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Monday, January 31, 2022 9:02 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
>
> On Mon, Jan 31, 2022 at 1:08 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Mon, Jan 31, 2022 at 7:27 AM houzj(dot)fnst(at)fujitsu(dot)com
> > <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> > >
> > > On Monday, January 31, 2022 8:53 AM Peter Smith
> <smithpb2250(at)gmail(dot)com> wrote:
> > > >
> > > > PSA v73*.
> > > >
> > > > (A rebase was needed due to recent changes in tab-complete.c.
> > > > Otherwise, v73* is the same as v72*).
> > >
> > > Thanks for the rebase.
> > > Attach the V74 patch set which did the following changes:
> > >
> >
> > Few comments:
> > =============
> >
>
> Few more minor comments:
> 1.
> + if (relentry->attrmap)
> + {
> + TupleDesc tupdesc = RelationGetDescr(relation); TupleTableSlot
> + *tmp_slot = MakeTupleTableSlot(tupdesc,
> + &TTSOpsVirtual);
> +
> + new_slot = execute_attr_map_slot(relentry->attrmap,
> + new_slot,
> + tmp_slot);
>
> I think we don't need these additional variables tupdesc and tmp_slot.
> You can directly use MakeTupleTableSlot instead of tmp_slot, which will make
> this and nearby code look better.

Changed.

> 2.
> + if (pubrinfo->pubrelqual)
> + appendPQExpBuffer(query, " WHERE (%s)", pubrinfo->pubrelqual);
> + appendPQExpBufferStr(query, ";\n");
>
> Do we really need additional '()' for rwo filter expression here? See the below
> output from pg_dump:
>
> ALTER PUBLICATION pub1 ADD TABLE ONLY public.t1 WHERE ((c1 < 100));

I will investigate this and change this later if needed.

> 3.
> + /* row filter (if any) */
> + if (pset.sversion >= 150000)
> + {
> + if (!PQgetisnull(result, i, 1))
> + appendPQExpBuffer(&buf, " WHERE %s", PQgetvalue(result, i, 1)); }
>
> I don't think we need this version check if while forming query we use NULL as
> the second column in the corresponding query for v < 150000.

Changed.

Attach the V75 patch set which address the above, Amit's[1] and Greg's[2][3] comments.

The new version patch also includes the following changes:

- run pgindent
- adjust some comments
- remove some unnecessary ExecClearTuple
- slightly improve the row filter of toast case by removing some unnecessary
memory allocation and directly return the modified new slot instead of
copying it again.

[1] https://www.postgresql.org/message-id/CAA4eK1LjyiPkwOki3n%2BQfORmBQLUvsvBfifhZMh%2BquAJTuRU_w%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CAJcOf-fR_BKHNuz7AXCWuk40ESVOr%3DDkXf3evbNNi4M4V_5agQ%40mail.gmail.com
[3] https://www.postgresql.org/message-id/CAJcOf-fR_BKHNuz7AXCWuk40ESVOr%3DDkXf3evbNNi4M4V_5agQ%40mail.gmail.com

Best regards,
Hou zj

Attachment Content-Type Size
v75-0002-Row-filter-tab-auto-complete-and-pgdump.patch application/octet-stream 5.8 KB
v75-0000-clean-up-pgoutput-cache-invalidation.patch application/octet-stream 7.0 KB
v75-0001-Allow-specifying-row-filters-for-logical-replication.patch application/octet-stream 155.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message houzj.fnst@fujitsu.com 2022-02-01 03:57:30 RE: row filtering for logical replication
Previous Message Fujii Masao 2022-02-01 03:40:41 Re: Add checkpoint and redo LSN to LogCheckpointEnd log message