Skip site navigation (1) Skip section navigation (2)

Re: WIP: Fast GiST index build

From: Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
To: Alexander Korotkov <aekorotkov(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: WIP: Fast GiST index build
Date: 2011-08-30 17:29:13
Message-ID: 4E5D1DE9.8070502@enterprisedb.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On 30.08.2011 13:29, Alexander Korotkov wrote:
> On Tue, Aug 30, 2011 at 1:13 PM, Heikki Linnakangas<
> heikki(dot)linnakangas(at)enterprisedb(dot)com>  wrote:
>
>> So, over 50% of the CPU time is spent in choosing a block from the
>> temporary files. That should be pretty easy to improve..
>>
> Yes, probably we can just remove free blocks sorting.

Ok, the first results are in for that:

          testname          |   nrows   |    duration     | accesses
---------------------------+-----------+-----------------+----------
  points unordered buffered | 250000000 | 06:00:23.707579 |  4049832

 From the previous test runs, the unbuffered index build took under 4 
hours, so even though this is a lot better than with the sorting, it's 
still a loss compared to non-buffered build.

I had vmstat running during most of this index build. At a quick glance, 
it doesn't seem to be CPU bound anymore. I suspect the buffers in the 
temporary file gets very fragmented. Or, we're reading it in backwards 
order because the buffers work in a LIFO fashion. The system seems to be 
doing about 5 MB/s of I/O during the build, which sounds like a figure 
you'd get for more or less random I/O.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

In response to

Responses

pgsql-hackers by date

Next:From: Tom LaneDate: 2011-08-30 18:25:12
Subject: Re: Inputting relative datetimes
Previous:From: Robert HaasDate: 2011-08-30 16:50:19
Subject: Re: dropdb and dropuser: IF EXISTS

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group