Re: Hash join in SELECT target list expression keeps consuming memory

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Hash join in SELECT target list expression keeps consuming memory
Date: 2018-03-16 19:17:43
Message-ID: 6855.1521227863@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com> writes:
> If the SELECT target list expression is a join subquery, and if the
> subquery does a hash join, then the query keeps on consuming more and
> more memory. Below is such a query :

Thanks for the report!

I dug into this with valgrind, and found that the problem is that
ExecHashTableCreate allocates some memory that isn't freed by
ExecHashTableDestroy, specifically the per-hash-key function
information. This is just dumb. We can keep that stuff in the
hashtable's hashCxt instead, where it will get freed at the right time.
The attached patch seems to fix it just by reordering the code.

I'm surprised nobody's noticed this before; maybe the problem is
of relatively recent vintage? Haven't checked the back branches yet.

regards, tom lane

Attachment Content-Type Size
fix-hashtable-memory-leak.patch text/x-diff 2.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2018-03-16 20:01:58 Re: Update doc links to https where appropriate?
Previous Message Andres Freund 2018-03-16 18:54:27 Update doc links to https where appropriate?