RE: logical decoding and replication of sequences, take 2

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Cc: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Subject: RE: logical decoding and replication of sequences, take 2
Date: 2023-11-27 12:08:24
Message-ID: TY3PR01MB9889DF7B06484285501A563FF5BDA@TY3PR01MB9889.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dear Amit, Tomas,

> > >
> > > I am wondering that instead of building the infrastructure to know
> > > whether a particular change is transactional on the decoding side,
> > > can't we have some flag in the WAL record to note whether the change
> > > is transactional or not? I have discussed this point with my colleague
> > > Kuroda-San and we thought that it may be worth exploring whether we
> > > can use rd_createSubid/rd_newRelfilelocatorSubid in RelationData to
> > > determine if the sequence is created/changed in the current
> > > subtransaction and then record that in WAL record. By this, we need to
> > > have additional information in the WAL record like XLOG_SEQ_LOG but we
> > > can probably do it only with wal_level as logical.
> > >
> >
> > I may not understand the proposal exactly, but it's not enough to know
> > if it was created in the same subxact. It might have been created in
> > some earlier subxact in the same top-level xact.
> >
>
> We should be able to detect even some earlier subxact or top-level
> xact based on rd_createSubid/rd_newRelfilelocatorSubid.

Here is a small PoC patchset to help your understanding. Please see attached
files.

0001, 0002 were not changed, and 0004 was reassigned to 0003.
(For now, I focused only on test_decoding, because it is only for evaluation purpose.)

0004 is what we really wanted to say. is_transactional is added in WAL record, and it stores
whether the operations is transactional. In order to distinguish the status, rd_createSubid and
rd_newRelfilelocatorSubid are used. According to the comment, they would be a valid value
only when the relation was changed within the transaction
Also, sequences_hash was not needed anymore, so it and related functions were removed.

How do you think?

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

Attachment Content-Type Size
v202311272-0001-Logical-decoding-of-sequences.txt text/plain 68.5 KB
v202311272-0002-Add-decoding-of-sequences-to-test_decodin.txt text/plain 20.4 KB
v202311272-0003-tweak-ReorderBufferSequenceIsTransactiona.txt text/plain 4.9 KB
v202311272-0004-WIP-add-is_transactional-attribute-in-xl_.txt text/plain 25.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Zhijie Hou (Fujitsu) 2023-11-27 12:23:20 RE: Synchronizing slots from primary to standby
Previous Message Drouvot, Bertrand 2023-11-27 12:04:41 Re: Synchronizing slots from primary to standby