Re: Should pg 11 use a lot more memory building an spgist index?

Bruno Wolff III <bruno(at)wolff(dot)to> writes:
> I have something that seems to produce it on rhel7. Fedora isn't working
> well either, but the difference may be due to postgresql.conf being
> different or some difference in the Fedora build.

Hmm, in my hands this produces the same size leak (~28GB) in either v10
or v11. In HEAD, somebody's made it even worse (~43GB). So this is
certainly pretty broken, but I'm not sure why it seems worse to you in
v11 than before.

The core of the problem I see is that check_exclusion_or_unique_constraint
does index_beginscan/index_rescan/index_endscan in a long-lived context,
while spgendscan seems to have employed dice while deciding which of
spgbeginscan's allocations it would bother to pfree. This is ancient,
though the specific case you have here can only be tested back to v10
because the inet SPGIST opclass wasn't there before.

A quick review of the other index AM endscan methods seems to indicate
that they all try to clean up their mess. So maybe we should just make
spgendscan do likewise. Alternatively, we could decide that requiring
endscan methods to free storage is not very safe, in which case it would
be incumbent on check_exclusion_or_unique_constraint to make a temporary
context to run the scan in. But if we're going to do the latter, I think
we oughta go full in and remove the retail pfree's from all the *endscan
functions. We'd also have to review other callers of
index_beginscan/index_endscan to see which ones might also need their own
temp contexts. So that would surely end up being more invasive than
just adding some pfree's to spgendscan would be. Maybe in the long run
it'd be worth it, but probably not in the short run, or for back-patching.

Thoughts?

regards, tom lane