| From: | shveta malik <shveta(dot)malik(at)gmail(dot)com> |
|---|---|
| To: | Dilip Kumar <DilipBalaut(at)gmail(dot)com> |
| Cc: | Zhijie Hou <houzhijie22(at)gmail(dot)com>, Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, saurabh singh <saurabh(dot)singh214(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com> |
| Subject: | Re: Proposal: Conflict log history table for Logical Replication |
| Date: | 2026-10-09 09:43:30 |
| Message-ID: | CAJpy0uB7ExJjtgsSipnGOZT6p=kpx3OWaXSph+KRNaQKrhvCLw@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Another concern on v79 is:
I noticed that one CLT row contains three timestamp representations
with three different behaviors:
1) replica_identity is frozen in the apply worker’s DateStyle.
Session's Datestyle has no impact on it.
2) remote_commit_ts is represented in the reader’s (session's) DateStyle, and
3) local_conflicts.commit_ts is always ISO.
This may cause some conufsion. Please see the results below. I can
share the testcase if needed:
Conflicts:
postgres=# select conflict_type, remote_commit_ts, replica_identity,
local_conflicts from pg_conflict.pg_conflict_log_16390;
conflict_type | remote_commit_ts |
replica_identity |
local_conflicts
-----------------------+--------------------------------+-----------------------------------+------------------------------------------------------------------------------------------
delete_origin_differs | 09/10/2026 14:15:29.055423 IST |
{"k":"2025-12-25 10:00:00+05:30"} |
{"{\"xid\":\"672\",\"commit_ts\":\"2026-10-09T14:15:22.033391+05:30\",\"origin\":null}"}
delete_origin_differs | 09/10/2026 14:17:12.929024 IST |
{"k":"02/01/2025 08:00:00 IST"} |
{"{\"xid\":\"678\",\"commit_ts\":\"2026-10-09T14:17:09.326423+05:30\",\"origin\":null}"}
(2 rows)
1) First conflict row was inserted with default Datastyle of DB and
the apply worker which is:
postgres=# show DateStyle;
DateStyle
-----------
ISO, DMY
2) Before inserting second conflict, below was done.
ALTER DATABASE postgres SET DateStyle = 'SQL, DMY';
ALTER SUBSCRIPTION sub1 DISABLE; ALTER SUBSCRIPTION sub1 ENABLE;
(So Above conflict data is read when session's format was also 'SQL,
DMY', thus we see remote_commit_ts in SQL format)
----
We can see the differnce in 'replica_identity' values in 2 rows, while
remote_commit_ts and local_conflicts.commit_ts remain same. Second RI
is 02/01/2025 in DMY format, meaning 2nd january.
3)
Now if I do this:
SET DateStyle = 'ISO, MDY'
postgres=# select conflict_type, remote_commit_ts, replica_identity,
local_conflicts from pg_conflict.pg_conflict_log_16390;
conflict_type | remote_commit_ts |
replica_identity |
local_conflicts
-----------------------+----------------------------------+-----------------------------------+------------------------------------------------------------------------------------------
delete_origin_differs | 2026-10-09 14:15:29.055423+05:30 |
{"k":"2025-12-25 10:00:00+05:30"} |
{"{\"xid\":\"672\",\"commit_ts\":\"2026-10-09T14:15:22.033391+05:30\",\"origin\":null}"}
delete_origin_differs | 2026-10-09 14:17:12.929024+05:30 |
{"k":"02/01/2025 08:00:00 IST"} |
{"{\"xid\":\"678\",\"commit_ts\":\"2026-10-09T14:17:09.326423+05:30\",\"origin\":null}"}
(2 rows)
See remote_commit_ts changed from previous SQL to ISO, while
replica_identity and local_conflicts.commit_ts are the same.
If I try to read it as documented (casting it back to original
column's type), I get misleading results:
postgres=# SET DateStyle = 'ISO, MDY';
postgres=#
SELECT (local_conflicts[1]->>'commit_ts')::timestamptz as local_ts,
(replica_identity->>'k')::timestamptz as ri_ts
FROM pg_conflict.pg_conflict_log_16390;
local_ts | ri_ts
----------------------------------+---------------------------
2026-10-09 14:15:22.033391+05:30 | 2025-12-25 10:00:00+05:30
2026-10-09 14:17:09.326423+05:30 | 2025-02-01 08:00:00+05:30
(2 rows)
See second row for ri_ts, it will now represent '1st Feb' rather than
'2nd January'.
Shouldn't all three timestamps be in the same format?
IMO, remote_commit_ts being session-dependent is expected.
The concern is the other two JSON text columns. replica_identity
stores the worker’s DateStyle, so the ->>'k'::timestamptz can
silently return a different value (here 2nd Jan read back as 1st Feb);
while local_conflicts.commit_ts is always ISO. Shouldn’t the recorded
JSON timestamps use a single, stable format (ISO) so they’re
unambiguous and safe to cast back?
thanks
Shveta
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Nikhil Sontakke | 2026-10-09 09:46:10 | Re: Publication DDL can race with a concurrent UPDATE |
| Previous Message | Shubhra Jain | 2026-10-09 09:18:58 | Looking for a good first patch to author |