| From: | "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com> |
|---|---|
| To: | 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>, Duncan Sands <duncan(dot)sands(at)deepbluecap(dot)com> |
| Cc: | "pgsql-bugs(at)lists(dot)postgresql(dot)org" <pgsql-bugs(at)lists(dot)postgresql(dot)org>, Sawada Masahiko <sawada(dot)mshk(at)gmail(dot)com> |
| Subject: | RE: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5 |
| Date: | 2025-05-21 11:48:12 |
| Message-ID: | OSCPR01MB14966139B6C89C27647712A90F59EA@OSCPR01MB14966.jpnprd01.prod.outlook.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-bugs |
Dear hackers,
> I think the problem here is that when we are distributing
> invalidations to a concurrent transaction, in addition to queuing the
> invalidations as a change, we also copy the distributed invalidations
> along with the original transaction's invalidations via repalloc in
> ReorderBufferAddInvalidations. So, when there are many in-progress
> transactions, each would try to copy all its accumulated invalidations
> to the remaining in-progress transactions. This could lead to such an
> increase in allocation request size. However, after queuing the
> change, we don't need to copy it along with the original transaction's
> invalidations. This is because the copy is only required when we don't
> process any changes in cases like ReorderBufferForget(). I have
> analyzed all such cases, and my analysis is as follows:
Based on the analysis, I created a PoC which avoids the repalloc().
Invalidation messages distributed by SnapBuildDistributeSnapshotAndInval() are
skipped to add in the list, just queued - repalloc can be skipped. Also, the function
distributes messages only in the list, so received messages won't be sent again.
Now a patch for PG17 is created for testing purpose. Duncan, can you apply this and
confirms whether the issue can be solved?
Best regards,
Hayato Kuroda
FUJITSU LIMITED
| Attachment | Content-Type | Size |
|---|---|---|
| PG17-0001-Avoid-distributing-invalidation-messages-several-tim.patch | application/octet-stream | 5.0 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Duncan Sands | 2025-05-21 13:27:31 | Re: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5 |
| Previous Message | Duncan Sands | 2025-05-21 11:30:58 | Re: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5 |