Re: Proposal: Conflict log history table for Logical Replication

From: shveta malik <shveta(dot)malik(at)gmail(dot)com>
To: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Cc: Peter Smith <smithpb2250(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, shveta malik <shveta(dot)malik(at)gmail(dot)com>
Subject: Re: Proposal: Conflict log history table for Logical Replication
Date: 2026-01-29 11:16:10
Message-ID: CAJpy0uCvQuC+x7OV6aUv5MWHcVMOqOdDcnCK5uN=d7VKthpY8A@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dilip, I was testing patch002, few observations:

1)
remote_commit_ts is '2026-01-29 14:38:02.777599+05:30' while
local_conflicts.commit_ts is of format
'2026-01-29T14:37:35.805819+05:30' i.e. a 'T' to separate date and
time in local-commit_ts case. IIUC, this is the expected (standard)
way for timestamps to be in json-format. Can you please confirm?

2)
Differences between LOG and TABLE:

a) In LOG, the column names are not included in the remote or local
tuples, whereas in the TABLE, they are. For example:

LOG: Could not apply remote change: remote row (2, 3, 4)
TABLE: remote_tuple: {"a":2,"b":3,"c":4}

b) remote_commit_ts is not recorded in LOG, but it is included in the TABLE.

IMO, both of these differences are okay as TABLE is expected to store
(or can store) more detailed information. I’m just noting these here
for others’ input. Let me know your thoughts.

3)
If we look at the table overall, the name 'local_conflicts' can be a
bit misleading. This column actually stores the details about local
rows involved in a conflict, rather than describing the conflict
itself. See example below. A clearer name might be local_data,
local_tuples_data, or local_tuples_detail. Thoughts?

local_conflicts for one update_exists:
{"{\"xid\":\"791\",\"commit_ts\":\"2026-01-29T14:37:35.805819+05:30\",\"origin\":null,\"key\":{\"i\":40},\"tuple\":{\"i\":40,\"j\":10}}"}
~~

patch002 works well in my testing done so far.

thanks
Shveta

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Mircea Cadariu 2026-01-29 11:19:05 Re: parallel data loading for pgbench -i
Previous Message Ilia Evdokimov 2026-01-29 11:14:43 Re: Hash-based MCV matching for large IN-lists