Re: Should pg 11 use a lot more memory building an spgist index?

From: Bruno Wolff III <bruno(at)wolff(dot)to>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: pgsql-general(at)lists(dot)postgresql(dot)org, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Should pg 11 use a lot more memory building an spgist index?
Date: 2018-10-26 11:23:14
Message-ID: 20181026112314.GA8199@wolff.to
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Fri, Oct 26, 2018 at 10:16:09 +0100,
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>Bruno Wolff III <bruno(at)wolff(dot)to> writes:
>> I have something that seems to produce it on rhel7. Fedora isn't working
>> well either, but the difference may be due to postgresql.conf being
>> different or some difference in the Fedora build.
>
>Hmm, in my hands this produces the same size leak (~28GB) in either v10
>or v11. In HEAD, somebody's made it even worse (~43GB). So this is
>certainly pretty broken, but I'm not sure why it seems worse to you in
>v11 than before.

As a short term work around, could I create the index first and use
insert statements, each in their own transaction, to get the table loaded
with the index?

Is the issue on Fedora taking very long to build a normal spgist index for
network addresses worth pursuing separately, or is it likely to be the same
underlying cause? I don't really need to get this working there, as that was
just to help with testing.

I could also try adjusting memory limits temporarily. If the leak is 28GB
on a 32GB system I might be able to get the index built if less memory
is tied up in other things. My workstation also has 32GB and the exclude
index did build there with the postgresql.conf having lower memory limits.

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Arup Rakshit 2018-10-26 11:54:04 ERROR: type "myrowtype" does not exist
Previous Message Олег Самойлов 2018-10-26 11:04:07 Re: Strange behavior of the random() function

Browse pgsql-hackers by date

  From Date Subject
Next Message Haribabu Kommi 2018-10-26 11:24:56 Re: Pluggable Storage - Andres's take
Previous Message Julien Rouhaud 2018-10-26 11:01:30 Re: Ordered Partitioned Table Scans