RE: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>, Duncan Sands <duncan(dot)sands(at)deepbluecap(dot)com>
Cc: "pgsql-bugs(at)lists(dot)postgresql(dot)org" <pgsql-bugs(at)lists(dot)postgresql(dot)org>, Sawada Masahiko <sawada(dot)mshk(at)gmail(dot)com>
Subject: RE: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5
Date: 2025-05-21 11:48:12
Message-ID: OSCPR01MB14966139B6C89C27647712A90F59EA@OSCPR01MB14966.jpnprd01.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Dear hackers,

> I think the problem here is that when we are distributing
> invalidations to a concurrent transaction, in addition to queuing the
> invalidations as a change, we also copy the distributed invalidations
> along with the original transaction's invalidations via repalloc in
> ReorderBufferAddInvalidations. So, when there are many in-progress
> transactions, each would try to copy all its accumulated invalidations
> to the remaining in-progress transactions. This could lead to such an
> increase in allocation request size. However, after queuing the
> change, we don't need to copy it along with the original transaction's
> invalidations. This is because the copy is only required when we don't
> process any changes in cases like ReorderBufferForget(). I have
> analyzed all such cases, and my analysis is as follows:

Based on the analysis, I created a PoC which avoids the repalloc().
Invalidation messages distributed by SnapBuildDistributeSnapshotAndInval() are
skipped to add in the list, just queued - repalloc can be skipped. Also, the function
distributes messages only in the list, so received messages won't be sent again.

Now a patch for PG17 is created for testing purpose. Duncan, can you apply this and
confirms whether the issue can be solved?

Best regards,
Hayato Kuroda
FUJITSU LIMITED

Attachment Content-Type Size
PG17-0001-Avoid-distributing-invalidation-messages-several-tim.patch application/octet-stream 5.0 KB

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Duncan Sands 2025-05-21 13:27:31 Re: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5
Previous Message Duncan Sands 2025-05-21 11:30:58 Re: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5