Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

From: li jie <ggysxcq(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
Date: 2023-12-01 08:25:17
Message-ID: CAGfChW5Qo2SrjJ7rU9YYtZbRaWv6v-Z8MJn=dQNx4uCSqDEOHA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> This is just an immature idea. I haven't started to implement it yet.
> Maybe it was designed this way because there
> are key factors that I didn't consider. So I want to hear everyone's
> opinions, especially the designers of logic decoding.

Attached is the patch I used to implement this optimization.
The main designs are as follows:
1. Added a callback LogicalDecodeFilterByRelCB for the output plugin.

2. Added this callback function pgoutput_table_filter for the pgoutput plugin.
Its main implementation is based on the table filter in the
pgoutput_change function.

3. After constructing a change and before Queue a change into a transaction,
use RelidByRelfilenumber to obtain the relation associated with the change,
just like obtaining the relation in the ReorderBufferProcessTXN function.

4. Relation may be a toast, and there is no good way to get its real
table relation
based on toast relation. Here, I get the real table oid through
toast relname, and
then get the real table relation.

5. This filtering takes into account INSERT/UPDATE/INSERT. Other
changes have not
been considered yet and can be expanded in the future.

6. The table filter in pgoutput_change and the get relation in
ReorderBufferProcessTXN
can be deleted. This has not been done yet. This is the next step.

Sincerely look forward to your feedback.
Regards, lijie

Attachment Content-Type Size
v1-Filter-irrelevant-change-before-reassemble-transacti.patch application/octet-stream 17.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Nazir Bilal Yavuz 2023-12-01 08:30:08 Re: Show WAL write and fsync stats in pg_stat_io
Previous Message Masahiko Sawada 2023-12-01 08:20:08 Re: [PoC] pg_upgrade: allow to upgrade publisher node