RE: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Masahiko Sawada' <sawada(dot)mshk(at)gmail(dot)com>
Cc: 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>, 'Duncan Sands' <duncan(dot)sands(at)deepbluecap(dot)com>, "'pgsql-bugs(at)lists(dot)postgresql(dot)org'" <pgsql-bugs(at)lists(dot)postgresql(dot)org>, 'vignesh C' <vignesh21(at)gmail(dot)com>
Subject: RE: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5
Date: 2025-06-06 06:54:51
Message-ID: OSCPR01MB1496608B31715BB094C194B70F56EA@OSCPR01MB14966.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Dear Sawada-san,

Sorry, I mistakenly sent partial part of the post. Let me continue the reply to 2.

> > 2.
> > If we add tests which is usable only for debug build, it must be run only when it
> > is enabled. IIUC such test does not exist yet.
>
> I think we need to test cases not to check if we reach a specific code
> point but to check if we can get the correct results even if we've
> executed various code paths. As for this bug, it is better to check
> that it works properly in a variety of cases. That way, we can check
> overflow cases and non-overflow cases also in test cases added in the
> future, improving the test coverage more.

You meant that 1) we do not have to ensure we reached the overflow part by seeing
the actual log output, and 2) it should be tested by existing ones.

Based on your advice, I updated the patch set.

0001 contains changes raised by [1]. I checked then and looked good.
0002 reduces the limitation to extremely lower value. I confirmed by adding debug
log and 7 cases can cause the overflow.

Appending
=======
Added debug log:
```
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3607,6 +3607,8 @@ ReorderBufferAddDistributedInvalidations(ReorderBuffer *rb, TransactionId xid,
*/
txn->txn_flags |= RBTXN_DISTR_INVAL_OVERFLOWED;

+ elog(LOG, "RBTXN_DISTR_INVAL_OVERFLOWED is set to the transaction")
```

Method how I count the testcases:
```
testrun$ grep -rI "RBTXN_DISTR_INVAL_OVERFLOWED" | awk '{print $1 $6}' | sort -u
subscription/100_bugs/log/100_bugs_twoways.log:2025-06-06LOG:
test_decoding/isolation/log/postmaster.log:2025-06-06isolation/catalog_change_snapshot/s1
test_decoding/isolation/log/postmaster.log:2025-06-06isolation/concurrent_ddl_dml/s2
test_decoding/isolation/log/postmaster.log:2025-06-06isolation/concurrent_stream/s1
test_decoding/isolation/log/postmaster.log:2025-06-06isolation/invalidation_distribution/s2
test_decoding/isolation/log/postmaster.log:2025-06-06isolation/oldest_xmin/s0
test_decoding/isolation/log/postmaster.log:2025-06-06isolation/snapshot_transfer/s0
```

[1]: https://www.postgresql.org/message-id/CAD21AoDaCL9X4E8VAe%3DfYa%3DzjqGTKRJW13dTPazqAuOAEEykOg%40mail.gmail.com

Best regards,
Hayato Kuroda
FUJITSU LIMITED

Attachment Content-Type Size
v12-0001-Fix-exponential-memory-allocation-issue-in-logic.patch application/octet-stream 13.9 KB
v12-0002-Make-MAX_DISTR_INVAL_MSG_PER_TXN-lower-in-case-o.patch application/octet-stream 1.2 KB

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Jim Jones 2025-06-06 10:22:30 Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL
Previous Message PG Bug reporting form 2025-06-06 06:39:26 BUG #18947: TRAP: failed Assert("len_to_wrt >= 0") in pg_stat_statements