Skip site navigation (1) Skip section navigation (2)

Re: build multiple indexes in single table pass?

From: ITAGAKI Takahiro <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>
To: Toru SHIMOGAKI <shimogaki(dot)toru(at)oss(dot)ntt(dot)co(dot)jp>, Andrew Dunstan <andrew(at)dunslane(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: build multiple indexes in single table pass?
Date: 2008-04-02 01:48:53
Message-ID: 20080402102030.9504.52131E4D@oss.ntt.co.jp (view raw or flat)
Thread:
Lists: pgsql-hackers
Toru SHIMOGAKI <shimogaki(dot)toru(at)oss(dot)ntt(dot)co(dot)jp> wrote:

> Andrew Dunstan wrote:
> > we could get a performance gain from building multiple indexes from a 
> > single sequential pass over the base table?
> 
> It is already implemented in pg_bulkload 
> (http://pgbulkload.projects.postgresql.org/).

I think there are two ways to implement multiple index creation.
  1. Add multiple indexes AFTER data loading.
  2. Define multiple indexes BEFORE data loading.

pg_bulkload uses the 2nd way, but the TODO item seems to target
the 1st, right? -- Both are useful, though.

| Allow multiple indexes to be created concurrently, ideally via a
| single heap scan, and have pg_restore use it

In either case, we probably need to renovate ambuild interface.
I'm thinking to reverse the control of heap sequential scans;
Seq scan is done in ambuild for now, but it will be controlled in
an external loop in the new method.

Define a new IndexBulder interface, something like:
    interface IndexBuilder
    {
       addTuple(IndexTuple tuple);
       finishBuild();
    }
and make ambuild() to return an IndexBuilder instance implemented in each AM.

However, it cannot use multiple CPUs if indexes are built in one process.
A less granular method might be better for Postgres, like synchronized scans,
as already pointed out.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center



In response to

pgsql-hackers by date

Next:From: Andrew DunstanDate: 2008-04-02 02:02:30
Subject: Re: column level privileges
Previous:From: sanjay sharmaDate: 2008-04-02 01:48:34
Subject: Re: column level privileges

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group