Re: Optimizing ResouceOwner to speed up COPY

On 10/17/25 00:17, Tom Lane wrote:
> Tomas Vondra <tomas(at)vondra(dot)me> writes:
>> On 10/16/25 21:28, Tom Lane wrote:
>>> I was thinking of adding some temporary instrumentation, like
>>> just elog'ing whenever the count goes above 1, and seeing where
>>> you get hits during the regression tests. I'm prepared to believe
>>> this is worth doing, but it'd be nice to have more examples
>>> in mind.
>
>> I tried that, and that gives me ~30k log messages with (count > 1). But
>> that's a bit misleading, because a lot of that are the same "thing"
>> going from 1 to N, which produces N messages.
>
>> If I subtract all the COPY statements, loading data for regressison
>> tests, that leaves ~7500 cases. There's a lot of cases with count 2 or
>> 3, mostly simple queries. Even a simple "\d t" produces a bunch of such
>> messages.
>
> Okay, so we definitely do have other cases where the count will be
> more than 1.
>
>> There are some more extreme ones too. For example
>> select infinite_recurse();
>> produces
>> WARNING: RESOURCEOWNER: plancache reference 0x34555828 resource owner
>> Portal count 1340
>
> Makes me wonder if we shouldn't make the count int64, just to remove
> all worries about overflow. That'd be free on 64-bit machines ...
>

We may do that, if it's free. But I doubt there's any risk of overflow
in practice. If we really wanted to support that many entries, we
wouldn't be allocating the hash table using MemoryContextAllocZero().
That puts the maximum number of unique entries at ~50M.

>> But I had another idea - see how large the ResourceOwners get, which
>> would tell us how much "overhead" it really is. So I added logging into
>> ResourceOwnerDelete (without the patch), and with that regression tests
>> produce 113916 messages. And 113289 have the initial capacity 32, so
>> array only. From the remaining ~600, only 72 have capacity over 64.
>
> Ah, that's fairly convincing. Seems like we can move ahead with this.
>

Thanks for the feedback. I'll let it sit for a while, there's no rush to
get this committed.

regards

--
Tomas Vondra