Reducing relcache memory usage: deduping index shapes

From: Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>
Subject: Reducing relcache memory usage: deduping index shapes
Date: 2026-08-31 20:56:57
Message-ID: CAEze2WiPyruOtUOSyRUV8mQssjmYwno0M6hkxC_iUpH-=W8WcA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

This is a part of the work I discussed in my PGConf.DEV talk of this year [0].

Many of Index's allocated fields in RelationData have the same (or
practically the same) contents for many indexes. E.g. a btree index on
a bigint column will always have the same contents in rd_opfamily,
rd_opcintype, and rd_support, and (given the same opclass options in
each column) will have equivalent rd_supportinfo.

The attached patchset adds a deduplication layer into the relcache,
which makes sure we only allocate one set of (rd_opfamily,
rd_opcintype, rd_support, rd_supportinfo) for indexes with equivalent
key definitions (so, a matching number of key attributes, opclasses,
and AM).

Additionally, it includes a patch by Andres (polished by me) that adds
a proxy context, which reduces the the overhead of small and
long-lived allocations in (what we expect to be) small memory contexts
by forwarding the allocations to malloc (after wrapping the struct).

Earlier versions of the patch adjusted aset.c to accept smaller memory
context sizes, but I abandoned that approach in favour of Andres'
ProxyContext -- it can outsource most the complexities of memory
management to the system allocator.

Patches in this patchset:
0001/0002: prepare relcache for deduplication.
0003: implements the deduplication
0004: Andres' ProxyContext patch
0005: Use proxy context in relcache for 'index data'

Kind regards,

Matthias van de Meent
Databricks (https://www.databricks.com)

[0]: https://www.youtube.com/watch?v=Q4w8LFWOwPY

Attachment Content-Type Size
v1-0003-Deduplicate-some-index-attributes-in-the-relcache.patch application/octet-stream 41.1 KB
v1-0004-MemCTX-Add-minimal-proxy-context-type-that-just-d.patch application/octet-stream 16.8 KB
v1-0005-Relcache-Use-Proxy-context-for-index-info-context.patch application/octet-stream 1.2 KB
v1-0002-Move-allocations-of-relation-opclass-fields-to-In.patch application/octet-stream 5.4 KB
v1-0001-Mark-constant-opclass-related-fields-const-in-Rel.patch application/octet-stream 4.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2026-08-31 20:59:23 Re: Commit Sequence Numbers and Visibility
Previous Message Paul A Jungwirth 2026-08-31 20:55:51 Temporal foreign key actions