Re: Should pg 11 use a lot more memory building an spgist index?

From: Bruno Wolff III <bruno(at)wolff(dot)to>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: pgsql-general(at)lists(dot)postgresql(dot)org, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Should pg 11 use a lot more memory building an spgist index?
Date: 2018-10-26 14:54:37
Message-ID: 20181026145437.GA11390@wolff.to
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Fri, Oct 26, 2018 at 13:44:07 +0100,
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>Bruno Wolff III <bruno(at)wolff(dot)to> writes:
>
>> As a short term work around, could I create the index first and use
>> insert statements, each in their own transaction, to get the table loaded
>> with the index?
>
>Yes; it might also be that you don't even need to break it up into
>separate statements.

It was time to refresh the geolite data anyway so I tried this. I needed
to turn memory_overcommit back on (0) to avoid an error, but the load went OK
without the oom killer doing anything. So things are fully working again.

Thanks for your help.

>> Is the issue on Fedora taking very long to build a normal spgist index for
>> network addresses worth pursuing separately, or is it likely to be the same
>> underlying cause?
>
>This issue only applies if it was an exclusion constraint. If you saw
>slowness or bloat with a plain index, that would be worth investigating
>separately.

I'll start a seperate thread if I get something to reasonably ask about.
The current dataset is probably a lot larger then needed to demonstrate the
issue. The difference might be do to configuration or how Fedora built it.
And I'll want to compare back to version 10. In the end I'll probably ask
why it is slower in one case as opposed to the other and it might not even
be a real bug.

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Jeff Janes 2018-10-26 15:42:59 Re: Different memory allocation strategy in Postgres 11?
Previous Message Tom Lane 2018-10-26 14:20:02 Re: Should pg 11 use a lot more memory building an spgist index?

Browse pgsql-hackers by date

  From Date Subject
Next Message Dagfinn Ilmari =?utf-8?Q?Manns=C3=A5ker?= 2018-10-26 16:11:59 Re: [PATCH] Tab complete EXECUTE FUNCTION for CREATE (EVENT) TRIGGER
Previous Message Michael Paquier 2018-10-26 14:43:26 Re: [PATCH] Tab complete EXECUTE FUNCTION for CREATE (EVENT) TRIGGER