Re: Proposal: Conflict log history table for Logical Replication

From: vignesh C <vignesh21(at)gmail(dot)com>
To: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Cc: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, saurabh singh <saurabh(dot)singh214(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, shveta malik <shveta(dot)malik(at)gmail(dot)com>
Subject: Re: Proposal: Conflict log history table for Logical Replication
Date: 2026-08-21 06:43:36
Message-ID: CALDaNm0=5_2BMPhT=DSrSNKuREF70N2ne8RxEszBfCcQ43Tzug@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, 20 Aug 2026 at 14:34, Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> Thanks for the patches, I have merged my changes with your patch and
> also fixed this comment, and renamed GetConflictLogDestAndTable to
> get_conflictlog_dest_and_table as this is a static function.

Thanks for the patch Dilip.
Logical replication breaks when conflict_log_destination = 'table' and
an update_origin_differs conflict involves a large row.
When the conflict occurs on a row containing a large value (~190 MB),
the conflict log table path converts the old row data into a string
representation. During this conversion, the resulting string buffer
grows beyond 1 GB and exceeds PostgreSQL's maximum StringInfo buffer
size of 1073741823 bytes.

This results in:
"ERROR: string buffer exceeds maximum allowed length (1073741823 bytes)"
"DETAIL: Cannot enlarge string buffer containing 1073741808 bytes by
32 more bytes."

As a result, the logical replication apply worker exits, and
replication cannot make further progress.
Interestingly, the same scenario works successfully with
conflict_log_destination = 'log'. In that case, the conflict is logged
successfully and logical replication continues without any error.

Reproducer
The issue can be reproduced with the following steps:
1) Set up logical replication with conflict_log_destination = 'table'
and create the following table:
"CREATE TABLE tab (a int PRIMARY KEY, b text);"
2) On the subscriber, enable track_commit_timestamp = on to detect
update_origin_differs conflicts.
3) Insert a row on the publisher:
"INSERT INTO tab VALUES (1, 'small');"
4) On the subscriber, update the replicated row with a large value:
"UPDATE tab SET b = repeat(chr(1), 190_000_000) WHERE a = 1;"
5) On the publisher, update the same row to generate an
update_origin_differs conflict on the subscriber:
"UPDATE tab SET b = 'new' WHERE a = 1;"

The apply worker then fails with the following error:
"2026-08-21 11:37:57.104 IST [49531] LOG: background worker 'logical
replication apply worker' (PID 49545) exited with exit code 1
2026-08-21 11:38:02.957 IST [49806] ERROR: string buffer exceeds
maximum allowed length (1073741823 bytes)
2026-08-21 11:38:02.957 IST [49806] DETAIL: Cannot enlarge string
buffer containing 1073741808 bytes by 32 more bytes."

The apply worker continues to fail on subsequent restart attempts, so
logical replication does not make further progress.

Thanks,
Vignesh

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2026-08-21 06:50:23 Re: [PATCH] Fix compilation of nodeMergejoin.c with EXEC_MERGEJOINDEBUG
Previous Message Daniel Gustafsson 2026-08-21 06:28:52 Re: [PATCH] Several refactorings for pg_dump