Re: Reducing relcache memory usage: deduping index shapes

Hi Matthias!

> Because indexes allocate an array of nkeyatts * indam->amsupport
> FmgrInfos, which for some index shapes can result in huge allocations
> (BRIN: 15, GiST: 12, btree: 6). There is certainly space for further
> optimization in other places, but this required changes in fewer code
> areas than if I'd started changing the data types and shape of
> Relation itself, given the spread of Relation across the codebase.

I see.

>> On the testing end: if this effort makes more progress, I could run this
>> against the aforementioned database to get some real-world numbers.
>
> Partitioned tables are fairly common, and frequently have the
> equivalent index definitions that this patch optimizes. I'd love to
> see real-world data on this optimization, and given the lack of churn
> in this part of the code I'm fairly confident this can be applied on
> older versions for educational purposes.

The use case with partitioned tables makes a lot of sense.

The workload I have uses very few partitioned tables but has a huge
number of partially very wide unpartitioned tables with large numbers of
indexes. I'll collect baseline numbers without the patch and figure out
how we can get numbers with the patch.

>> One more micro optimization that we could do is better packing
>> RelationData. On my AMD64 system sizeof(RelationData) == 488 bytes. By
>> reordering the members we could get it down to 440 bytes which is about
>> 10% savings.
>
> Yeah, packing RelationData would save some in struct size, but given
> that these are mostly still allocated as separate allocations in an
> aset context, shaving bytes off of RelationData won't help much until
> we get below 256 bytes. I would love to get the struct size that far
> down, but I don't expect that to be achievable with the current
> contents of the struct.
>
> I've considered allocating Relations in a slab context to avoid aset's
> alignment overhead, but never got far enough with a prototype to get
> it to pass all tests.

With the other optimizations from [1] we should be able to get below 256
bytes. If you want I can give this a try.

--
David Geier

[1]
https://www.postgresql.org/message-id/2bd41932-24fa-477a-a213-02fda4555835%40gmail.com