Re: 8.3 / 8.2.6 restore comparison

From: andy <andy(at)squeakycode(dot)net>
To: Heikki Linnakangas <heikki(at)enterprisedb(dot)com>
Cc: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>, Luke Lonergan <llonergan(at)greenplum(dot)com>, Greg Smith <gsmith(at)gregsmith(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: 8.3 / 8.2.6 restore comparison
Date: 2008-02-24 14:21:18
Message-ID: 47C17D5E.7080305@squeakycode.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Heikki Linnakangas wrote:
> Joshua D. Drake wrote:
>> On Sun, 24 Feb 2008 00:43:18 +0000
>> "Heikki Linnakangas" <heikki(at)enterprisedb(dot)com> wrote:
>>
>>> Incidentally, I've been working on a patch to speed up CopyReadLine.
>>> I was going to run some more tests first, but since we're talking
>>> about it, I guess I should just post the patch. I'll post to
>>> pgsql-patches shortly.
>>
>> On your post to patches you mentioned only about a 5% improvement.
>> Don't get me wrong, 5% is 5% and I respect it greatly but as far as I
>> can tell we are about 300% behind the curve.
>
> Yeah. Looking at the profile, the time is spent really all over the
> place. There's no one clear bottleneck to focus on. I think we could do
> a few more ~5% improvements, but
>
> At some point, I think we have to bite the bullet and find a way to use
> multiple CPUs for a single load. I don't have any good ideas or plans
> for that, but hopefully someone does.
>

There was talk elsewhere about making pg_dump/restore smarter. It could
create tables, then COPY, and create the indexes last. Add to that
pg_restore using multiple connections and you'd have it. One connection
could do a COPY, then a second connection could be created to start the
CREATE INDEX's for that table, while the first connection went on to
COPY the next table.

Or something like that...

-Andy

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Florian G. Pflug 2008-02-24 16:49:02 Behaviour of rows containg not-null domains in plpgsql
Previous Message Jochem van Dieten 2008-02-24 13:22:32 Re: pg_dump additional options for performance