Re: Reducing relcache memory usage: deduping index shapes

On Tue, 1 Sept 2026 at 10:06, David Geier <geidav(dot)pg(at)gmail(dot)com> wrote:
>
> > The attached patchset adds a deduplication layer into the relcache,
> > which makes sure we only allocate one set of (rd_opfamily,
> > rd_opcintype, rd_support, rd_supportinfo) for indexes with equivalent
> > key definitions (so, a matching number of key attributes, opclasses,
> > and AM).
>
> Why did you specifically worked on deduplicating the index fields in
> RelationData? Is that consuming most out of all of RelationData?

Because indexes allocate an array of nkeyatts * indam->amsupport
FmgrInfos, which for some index shapes can result in huge allocations
(BRIN: 15, GiST: 12, btree: 6). There is certainly space for further
optimization in other places, but this required changes in fewer code
areas than if I'd started changing the data types and shape of
Relation itself, given the spread of Relation across the codebase.

> > Additionally, it includes a patch by Andres (polished by me) that adds
> > a proxy context, which reduces the the overhead of small and
> > long-lived allocations in (what we expect to be) small memory contexts
> > by forwarding the allocations to malloc (after wrapping the struct).
> >
> > Earlier versions of the patch adjusted aset.c to accept smaller memory
> > context sizes, but I abandoned that approach in favour of Andres'
> > ProxyContext -- it can outsource most the complexities of memory
> > management to the system allocator.
> >
> > Patches in this patchset:
> > 0001/0002: prepare relcache for deduplication.
> > 0003: implements the deduplication
> > 0004: Andres' ProxyContext patch
> > 0005: Use proxy context in relcache for 'index data'
>
> On the testing end: if this effort makes more progress, I could run this
> against the aforementioned database to get some real-world numbers.

Partitioned tables are fairly common, and frequently have the
equivalent index definitions that this patch optimizes. I'd love to
see real-world data on this optimization, and given the lack of churn
in this part of the code I'm fairly confident this can be applied on
older versions for educational purposes.

> One more micro optimization that we could do is better packing
> RelationData. On my AMD64 system sizeof(RelationData) == 488 bytes. By
> reordering the members we could get it down to 440 bytes which is about
> 10% savings.

Yeah, packing RelationData would save some in struct size, but given
that these are mostly still allocated as separate allocations in an
aset context, shaving bytes off of RelationData won't help much until
we get below 256 bytes. I would love to get the struct size that far
down, but I don't expect that to be achievable with the current
contents of the struct.

I've considered allocating Relations in a slab context to avoid aset's
alignment overhead, but never got far enough with a prototype to get
it to pass all tests.

Kind regards,

Matthias van de Meent
Databricks (https://www.databricks.com)