RE: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Masahiko Sawada' <sawada(dot)mshk(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Duncan Sands <duncan(dot)sands(at)deepbluecap(dot)com>, "pgsql-bugs(at)lists(dot)postgresql(dot)org" <pgsql-bugs(at)lists(dot)postgresql(dot)org>
Subject: RE: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5
Date: 2025-05-28 12:27:51
Message-ID: OSCPR01MB149667B316377CE0615E15138F567A@OSCPR01MB14966.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Dear Sawada-san, Amit,

> > It can impact the performance for large transactions with fewer
> > invalidations, especially the ones which has spilled changes because
> > it needs to traverse the entire list of changes again at the end.
>
> Agreed.
>
> > The
> > other idea would be to add new member(s) in ReorderBufferTXN to
> > receive distributed invalidations. For adding the new member in
> > ReorderBufferTXN: (a) in HEAD, it should be okay, (b) for
> > backbranches, we may be able to add at the end, but we should check if
> > there are any extensions using sizeof(ReorderBufferTxn) and if they
> > are using what we need to do.
>
> If we can make sure that that change won't break the existing
> extensions, I think this would be the most reasonable solution.

Based on the discussion, I created PoC for master/PG17. Please see attached.
The basic idea is to introduce the new queue which only contains distributed inval
messages. Contents are consumed at end of transactions. I feel some of codes can
be re-used so that internal functions are introduced. At least, it could pass
regression tests and workloads discussed here.

Best regards,
Hayato Kuroda
FUJITSU LIMITED

Attachment Content-Type Size
v4-PG17-0001-Avoid-distributing-invalidation-messages-sev.patch application/octet-stream 21.2 KB
v4-master-0001-Avoid-distributing-invalidation-messages-s.patch application/octet-stream 21.2 KB

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Tom Lane 2025-05-28 14:25:50 Re: [EXT] Re: GSS Auth issue when user member of lots of AD groups
Previous Message Chris Gooch 2025-05-28 10:36:46 RE: [EXT] Re: GSS Auth issue when user member of lots of AD groups