Re: Out of Memory errors are frustrating as heck!

Jeff Janes <jeff(dot)janes(at)gmail(dot)com> writes:
> The growEnabled stuff only prevents infinite loops. It doesn't prevent
> extreme silliness.

> If a single 32 bit hash value has enough tuples by itself to not fit in
> work_mem, then it will keep splitting until that value is in a batch by
> itself before shutting off

Right, that's the code's intention. If that's not good enough for this
case, we'll need to understand the details a bit better before we can
design a better(?) heuristic.

I suspect, however, that we might be better off just taking the existence
of the I/O buffers into account somehow while deciding whether it's worth
growing further. That is, I'm imagining adding a second independent
reason for shutting off growEnabled, along the lines of "increasing
nbatch any further will require an unreasonable amount of buffer memory".
The question then becomes how to define "unreasonable".

regards, tom lane