Avoiding memory leakage in jsonpath evaluation

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Avoiding memory leakage in jsonpath evaluation
Date: 2026-03-17 21:33:31
Message-ID: 569394.1773783211@sss.pgh.pa.us
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I got an off-list report that a query like this consumes
an unreasonable amount of memory:

SELECT jsonb_path_query((SELECT jsonb_agg(i) FROM generate_series(1,10000) i),
'$[*] ? (@ < $)');

For me, that eats about 6GB by the time it's done executing.
If that doesn't seem like a lot to you, just add another zero to the
generate_series call, and then it'll be more like 600GB, because the
leakage is O(N^2).

Admittedly, this isn't an especially useful query: its runtime is
also O(N^2), because that path expression basically requires us to
compare every element of the input JSON array to every other element.
But it's not cool that it leaks so much memory while at it.

I poked into this and found that the leakage is entirely composed of
"JsonValueList"s that are built during path evaluation and then just
left to rot until the end of jsonb_path_query(). We can fix it by
being careful to free those lists on the way out of each jsonpath
evaluation function that creates one. However, just doing that would
mean adding pfree overhead on top of palloc overhead, so I went a bit
further and reimplemented JsonValueList to be more compact and cheaper
to allocate/free. The attached seems to be a bit faster than the
existing code as well as not leaking so much memory. See the draft
commit message for more details.

regards, tom lane

Attachment Content-Type Size
v1-0001-Fix-transient-memory-leakage-in-jsonpath-evaluati.patch text/x-diff 37.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jonathan Gonzalez V. 2026-03-17 21:37:39 Re: Make PGOAUTHCAFILE in libpq-oauth work out of debug mode
Previous Message Andres Freund 2026-03-17 21:19:49 Re: Need help debugging SIGBUS crashes