Skip site navigation (1) Skip section navigation (2)

Re: GIN index creation extremely slow ?

From: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
To: Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: GIN index creation extremely slow ?
Date: 2006-06-26 21:48:13
Message-ID: Pine.GSO.4.63.0606270147270.2921@ra.sai.msu.su (view raw or flat)
Thread:
Lists: pgsql-hackers
gin uses maintenance_work_mem,so try to increase it and see dramatic
improvements

Oleg
On Mon, 26 Jun 2006, Stefan Kaltenbrunner wrote:

> on IRC somebody mentioned that it took >34h to greate a GIN index (on a
> tsvector) on a ~3 Million column table (wikipedia dump) with a
> reasonable speced box (AMD 3400+).
> After getting hold of a dump of said table (around 4,1GB in size) I
> managed to get the following timings:
>
> test=# CREATE INDEX idxFTI_idx ON wikipedia USING gist(vector);
> CREATE INDEX
> Time: 416122.896 ms
>
> so about 7 minutes - sounds very reasonable
>
> test=# CREATE INDEX idxFTI2_idx ON wikipedia USING gin(vector);
> CREATE INDEX
> Time: 52681605.101 ms
>
> ouch - that makes for a whoppy 14,6hours(!). During that time the box is
> completely CPU bottlenecked and during virtually no IO at all - (varing
> maintainance_work_mem does not seem to make any noticable difference).
>
> That box is a fast Dual Opteron 2.6Ghz with 8GB RAM and a 4 disk RAID10
> for the WAL and 12 disks for the data running a very recent -HEAD
> checkout ...
>
> It looks like we still don't have any docs for GIN in the tree so I
> don't know if those timings are expected or not ...
>
>
> Stefan
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>

 	Regards,
 		Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

In response to

pgsql-hackers by date

Next:From: Bruce MomjianDate: 2006-06-26 21:49:31
Subject: Re: [HACKERS] Overhead for stats_command_string et al, take
Previous:From: Tom LaneDate: 2006-06-26 21:43:22
Subject: Re: [HACKERS] Overhead for stats_command_string et al, take 2

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group