Re: 9.5: Better memory accounting, towards memory-bounded HashAgg

From: Peter Geoghegan <pg(at)heroku(dot)com>
To: Jeff Davis <pgsql(at)j-davis(dot)com>
Cc: Tomas Vondra <tv(at)fuzzy(dot)cz>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: 9.5: Better memory accounting, towards memory-bounded HashAgg
Date: 2014-12-28 20:45:42
Message-ID: CAM3SWZQnJ_xKex3YF6NkFmnTL9fYqBwODw+pbSiqpm3e_dL-+Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sun, Dec 28, 2014 at 12:37 PM, Jeff Davis <pgsql(at)j-davis(dot)com> wrote:
> Do others have similar numbers? I'm quite surprised at how little
> work_mem seems to matter for these plans (HashJoin might be a different
> story though). I feel like I made a mistake -- can someone please do a
> sanity check on my numbers?

I have seen external sorts that were quicker than internal sorts
before. With my abbreviated key patch, under certain circumstances
external sorts are faster, while presumably the same thing is true of
int4 attribute sorts today. Actually, I saw a 10MB work_mem setting
that was marginally faster than a multi-gigabyte one that fit the
entire sort in memory. It probably has something to do with caching
effects dominating over the expense of more comparisons, since higher
work_mem settings that still resulted in an external sort were slower
than the 10MB setting.

I was surprised by this too, but it has been independently reported by
Jeff Janes.
--
Peter Geoghegan

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2014-12-28 21:14:50 Re: 9.5: Memory-bounded HashAgg
Previous Message Jeff Davis 2014-12-28 20:37:05 Re: 9.5: Better memory accounting, towards memory-bounded HashAgg