Re: Reducing relcache memory usage: deduping index shapes

On Tue, 1 Sept 2026 at 08:57, Matthias van de Meent
<boekewurm+postgres(at)gmail(dot)com> wrote:
> Additionally, it includes a patch by Andres (polished by me) that adds
> a proxy context, which reduces the the overhead of small and
> long-lived allocations in (what we expect to be) small memory contexts
> by forwarding the allocations to malloc (after wrapping the struct).

Interesting. I looked at 0004 and expected that with a name like
"Proxy" that the allocation would be diverted to another context, such
as the parent context. If all pallocs are going directly to malloc,
would "Direct" not be a more suitable name?

Also, just so it's written down somewhere, can you elaborate on the
choice not to have the code similar to as it is, but instead of malloc
directly with MemoryContextAlloc in the parent context? Is it just a
case of problems with double counting for the memory stats? Is there
some other reason why this would be bad?

From a quick read of 0004:

1. The new context type should get a mention in
src/backend/utils/mmgr/README under "Alternative Memory Context
Implementations"

2. I think it's worth expanding the following to maybe tag something
like "i.e. are directly malloc'ed" or "i.e one malloc per palloc" to
the end. The whole thing about other context types managing oversized
chunks and directly mallocing a block for them is really up to them.

+ * Proxy is a MemoryContext implementation designed for memory usages which
+ * require their own memory context, but which generally have few allocations
+ * that generally have a very long lifetime. Compared to ASet, every
+ * allocation of a Proxy memory context gets an External chunk.

3. I don't quite understand the following comment. IMO, there is no
initial block here. This is just the malloc for the context struct
itself.

+ /*
+ * Allocate the initial block. Unlike other proxy.c blocks, it starts
+ * with the context header and its block header follows that.
+ */

4. Per the discussion in [1], I think the preference is to use size_t
instead of Size.

+ Size totalspace;
+ Size nchunks = 0;

Shouldn't nchunks be uint64 anyway? Nothing guarantees Size is bigger
than int, even on 64-bit.

5. The following WARNING looks buggy:

+ if (total_allocated != ctx->header.mem_allocated)
+ {
+ elog(WARNING, "problem in Proxy %s: amount of memory allocated %d
does not match header %d",
+ name, (int) total_allocated, ctx->chunks_allocated);
+ }

Why cast to int?
Why ctx->chunks_allocated and not ctx->header.mem_allocated?

> Earlier versions of the patch adjusted aset.c to accept smaller memory
> context sizes, but I abandoned that approach in favour of Andres'
> ProxyContext -- it can outsource most the complexities of memory
> management to the system allocator.

Can you share more about this choice? What are the advantages of this
new context type over doing something like modifying aset.c to allow
passing of a 0 maxBlockSize so that all chunks are external? If it
were done that way, the while loop at the end of
AllocSetContextCreateInternal() could calculate allocChunkLimit to be
0 and that would result in AllocSetAlloc() always going with the
AllocSetAllocLarge() path. I currently can't see beyond this only
saving the "if (size > set->allocChunkLimit)" precheck. Or is it a
case of AllocSetContext being overly large due to the freelist array?

Can you provide information about how much memory is being saved from 0004+0005?

David

[1] https://postgr.es/m/flat/CA%2BRLCQzSkLrwscci4%2Bu3eqymzbozXPdFt_TsU_dXPHvU4Px0dg%40mail.gmail.com