Re: Avoiding hash join batch explosions with extreme skew and weird stats

On Thu, May 16, 2019 at 06:58:43PM -0400, Tom Lane wrote:
>Thomas Munro <thomas(dot)munro(at)gmail(dot)com> writes:
>> On Fri, May 17, 2019 at 4:39 AM Tomas Vondra
>> <tomas(dot)vondra(at)2ndquadrant(dot)com> wrote:
>>> I kinda like the idea with increasing the spaceAllowed value. Essentially,
>>> if we decide adding batches would be pointless, increasing the memory
>>> budget is the only thing we can do anyway.
>
>> But that's not OK, we need to fix THAT.
>
>I don't think it's necessarily a good idea to suppose that we MUST
>fit in work_mem come what may. It's likely impossible to guarantee
>that in all cases. Even if we can, a query that runs for eons will
>help nobody.
>

I kinda agree with Thomas - arbitrarily increasing work_mem is something
we should not do unless abosolutely necessary. If the query is slow, it's
up to the user to bump the value up, if deemed appropriate.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services