Large table load (40 millon rows) - WAL hassles

From: simon(at)gonzo(dot)magellic(dot)com (simon lai)
To: pgsql-general(at)postgresql(dot)org
Subject: Large table load (40 millon rows) - WAL hassles
Date: 2001-07-04 23:42:32
Message-ID: slrn9k7af7.1cl.simon@gonzo.magellic.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Version: Postgres 7.1.2

A product we are developing requires frequent loading
of a large number of rows into a table. We are using
the "copy file" command, but with WAL we are effectively
doubling the amount of disk writing we are doing.

After the rows are loaded we do a "create index".

Is there a way to turn off WAL or otherwise speeding up
the table loading process? Would db_restore be faster?

The rows are loaded in sorted order. Does this impact
index creation negatively or positively?

We are currently working with test data but we estimate
production data to be 6 - 9 billion rows. Is anyone
else running with these volumes?

Simon

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2001-07-05 00:32:51 Re: sequencial scans
Previous Message Rob van Nieuwkerk 2001-07-04 23:13:38 Re: Red Hat Database Development