Re: Lots of memory allocated when reassigning Large Objects

From: Guillaume Lelarge <guillaume(at)lelarge(dot)info>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Justin Pryzby <pryzby(at)telsasoft(dot)com>, PostgreSQL Developers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Lots of memory allocated when reassigning Large Objects
Date: 2021-11-30 08:23:44
Message-ID: CAECtzeXcmQHDENgmMbY59ZMaDnm9vYo=NBMYDG=T3-TYyByEKQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Le mar. 30 nov. 2021 à 00:25, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> a écrit :

> Guillaume Lelarge <guillaume(at)lelarge(dot)info> writes:
> > Le lun. 29 nov. 2021 à 22:27, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> a écrit :
> >> I'm checking it in HEAD though; perhaps there's something else wrong
> >> in the back branches?
>
> > That's also what I was thinking. I was only trying with v14. I just
> checked
> > with v15devel, and your patch works alright. So there must be something
> > else with back branches.
>
> AFAICT the patch fixes what it intends to fix in v14 too. The reason the
> residual leak is worse in v14 is that the sinval message queue is bulkier.
> We improved that in HEAD in commit 3aafc030a.

I wanted to make sure that commit 3aafc030a fixed this issue. So I did a
few tests:

without 3aafc030a, without your latest patch
1182148 kB max resident size
with 3aafc030a, without your latest patch
1306812 kB max resident size

without 3aafc030a, with your latest patch
1182128 kB max resident size
with 3aafc030a, with your latest patch
180996 kB max resident size

Definitely, 3aafc030a and your latest patch allow PostgreSQL to use much
less memory. Going from 1GB to 180MB is awesome.

I tried to cherry-pick 3aafc030a on v14, but it didn't apply cleanly, and
the work was a bit overwhelming for me, at least in the morning. I'll try
again today, but I don't have much hope.

I'm not sure if I want to
> take the risk of back-patching that, even now that it's aged a couple
> months in the tree. It is a pretty localized fix, but it makes some
> assumptions about usage patterns that might not hold up.
>
>
I understand. Of course, it would be better if it could be fixed for each
supported version but I'm already happy that it could be fixed in the next
release.

--
Guillaume.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2021-11-30 08:29:12 Re: [PATCH] pg_statio_all_tables: several rows per table due to invalid TOAST index
Previous Message Masahiko Sawada 2021-11-30 07:45:55 Re: parallel vacuum comments