Re: Lots of memory allocated when reassigning Large Objects

On Mon, Nov 29, 2021 at 01:49:24PM +0100, Guillaume Lelarge wrote:
> One of our customers had a very bad issue while trying to reassign objects
> from user A to user B. He had a lot of them, and the backend got very
> hungry for memory. It finally all went down when the linux kernel decided
> to kill the backend (-9 of course).

> Memory is released at the end of the reassignment. So it's definitely not
> leaked forever, but only during the operation, which looks like a missing
> pfree (or something related). I've tried to find something like that in the
> code somewhere, but to no avail. I'm pretty sure I missed something, which
> is the reason for this email :)

I reproduced the issue like this.

psql postgres -c 'CREATE ROLE two WITH login superuser'
psql postgres two -c "SELECT lo_import('/dev/null') FROM generate_series(1,22111)" >/dev/null
psql postgres -c 'SET client_min_messages=debug; SET log_statement_stats=on;' -c 'begin; REASSIGN OWNED BY two TO pryzbyj; rollback;'

I didn't find the root problem, but was able to avoid the issue by creating a
new mem context. I wonder if there are a bunch more issues like this.

unpatched:
! 33356 kB max resident size

patched:
! 21352 kB max resident size

diff --git a/src/backend/catalog/pg_shdepend.c b/src/backend/catalog/pg_shdepend.c
index 9ea42f805f..cbe7f04983 100644
--- a/src/backend/catalog/pg_shdepend.c
+++ b/src/backend/catalog/pg_shdepend.c
@@ -65,6 +65,7 @@
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/fmgroids.h"
+#include "utils/memutils.h"
#include "utils/syscache.h"

typedef enum
@@ -1497,6 +1498,11 @@ shdepReassignOwned(List *roleids, Oid newrole)
while ((tuple = systable_getnext(scan)) != NULL)
{
Form_pg_shdepend sdepForm = (Form_pg_shdepend) GETSTRUCT(tuple);
+ MemoryContext cxt, oldcxt;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext, "shdepReassignOwned cxt",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcxt = MemoryContextSwitchTo(cxt);

/*
* We only operate on shared objects and objects in the current
@@ -1598,8 +1604,12 @@ shdepReassignOwned(List *roleids, Oid newrole)
elog(ERROR, "unexpected classid %u", sdepForm->classid);
break;
}
+
/* Make sure the next iteration will see my changes */
CommandCounterIncrement();
+
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(cxt);
}

systable_endscan(scan);