Re: Enforce work_mem per worker

On Sat, Nov 27, 2021 at 04:33:07PM +0000, Arne Roland wrote:
> Hello!
>
> Since I used a lot of my time chasing short lived processes eating away big chunks of memory in recent weeks, I am wondering about a decent way to go about this.
> The problem I am facing essentially relates to the fact that work_mem settings, while they are enforced per hash and sort node, aren't enforced globally.
> One common case, that causes this problem more frequently than a few years ago, is the partitionwise_join. If there are a lot of partitions hash joined, we get a lot of hash nodes, each one potentially consuming work_mem.

> While avoiding oom seems a big deal to me, my search didn't turn up previous hackers discussions about this. There is a good chance I am missing something here, so I'd appreciate any pointers.

Here's some pointers ;)

https://www.postgresql.org/message-id/flat/20190708164401.GA22387%40telsasoft.com
https://www.postgresql.org/message-id/flat/20191216215314.qvrtgxaz4m755geq%40development#75e9930ac2cd353a8036dc71e8f5e6f7
https://www.postgresql.org/message-id/flat/CAH2-WzmNwV%3DLfDRXPsmCqgmm91mp%3D2b4FvXNF%3DcCvMrb8YFLfQ%40mail.gmail.com
- I don't recall reading all of this last one before, and it's got interesting
historic value, so I'm reading it myself now...

--
Justin