From: | Jeff Davis <pgsql(at)j-davis(dot)com> |
---|---|
To: | pgsql-hackers(at)postgresql(dot)org |
Cc: | Tomas Vondra <tomas(dot)vondra(at)postgresql(dot)org>, David Rowley <dgrowleyml(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de> |
Subject: | Re: Make MemoryContextMemAllocated() more precise |
Date: | 2020-04-08 00:21:00 |
Message-ID: | 412a3fbf306f84d8d78c4009e11791867e62b87c.camel@j-davis.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Mon, 2020-03-16 at 11:45 -0700, Jeff Davis wrote:
> AllocSet allocates memory for itself in blocks, which double in size
> up
> to maxBlockSize. So, the current block (the last one malloc'd) may
> represent half of the total memory allocated for the context itself.
Narrower approach that doesn't touch memory context internals:
If the blocks double up in size to maxBlockSize, why not just create
the memory context with a smaller maxBlockSize? I had originally
dismissed this as a hack that could slow down some workloads when
work_mem is large.
But we can simply make it proportional to work_mem, which makes a lot
of sense for an operator like HashAgg that controls its memory usage.
It can allocate in blocks large enough that we don't call malloc() too
often when work_mem is large; but small enough that we don't overrun
work_mem when work_mem is small.
I have attached a patch to do this only for HashAgg, using a new entry
point in execUtils.c called CreateWorkExprContext(). It sets
maxBlockSize to 1/16th of work_mem (rounded down to a power of two),
with a minimum of initBlockSize.
This could be a good general solution for other operators as well, but
that requires a bit more investigation, so I'll leave that for v14.
The attached patch is narrow and solves the problem for HashAgg nicely
without interfering with anything else, so I plan to commit it soon for
v13.
Regards,
Jeff Davis
Attachment | Content-Type | Size |
---|---|---|
CreateWorkExprContext.patch | text/x-patch | 4.4 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Kyotaro Horiguchi | 2020-04-08 00:37:10 | Re: [HACKERS] Restricting maximum keep segments by repslots |
Previous Message | Andres Freund | 2020-04-08 00:12:49 | Re: 2pc leaks fds |