| From: | Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com> |
|---|---|
| To: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de> |
| Subject: | Reducing relcache memory usage: deduping index shapes |
| Date: | 2026-08-31 20:56:57 |
| Message-ID: | CAEze2WiPyruOtUOSyRUV8mQssjmYwno0M6hkxC_iUpH-=W8WcA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
This is a part of the work I discussed in my PGConf.DEV talk of this year [0].
Many of Index's allocated fields in RelationData have the same (or
practically the same) contents for many indexes. E.g. a btree index on
a bigint column will always have the same contents in rd_opfamily,
rd_opcintype, and rd_support, and (given the same opclass options in
each column) will have equivalent rd_supportinfo.
The attached patchset adds a deduplication layer into the relcache,
which makes sure we only allocate one set of (rd_opfamily,
rd_opcintype, rd_support, rd_supportinfo) for indexes with equivalent
key definitions (so, a matching number of key attributes, opclasses,
and AM).
Additionally, it includes a patch by Andres (polished by me) that adds
a proxy context, which reduces the the overhead of small and
long-lived allocations in (what we expect to be) small memory contexts
by forwarding the allocations to malloc (after wrapping the struct).
Earlier versions of the patch adjusted aset.c to accept smaller memory
context sizes, but I abandoned that approach in favour of Andres'
ProxyContext -- it can outsource most the complexities of memory
management to the system allocator.
Patches in this patchset:
0001/0002: prepare relcache for deduplication.
0003: implements the deduplication
0004: Andres' ProxyContext patch
0005: Use proxy context in relcache for 'index data'
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0003-Deduplicate-some-index-attributes-in-the-relcache.patch | application/octet-stream | 41.1 KB |
| v1-0004-MemCTX-Add-minimal-proxy-context-type-that-just-d.patch | application/octet-stream | 16.8 KB |
| v1-0005-Relcache-Use-Proxy-context-for-index-info-context.patch | application/octet-stream | 1.2 KB |
| v1-0002-Move-allocations-of-relation-opclass-fields-to-In.patch | application/octet-stream | 5.4 KB |
| v1-0001-Mark-constant-opclass-related-fields-const-in-Rel.patch | application/octet-stream | 4.3 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Jeff Davis | 2026-08-31 20:59:23 | Re: Commit Sequence Numbers and Visibility |
| Previous Message | Paul A Jungwirth | 2026-08-31 20:55:51 | Temporal foreign key actions |