Skip site navigation (1) Skip section navigation (2)

Re: parallel pg_restore - WIP patch

From: Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: parallel pg_restore - WIP patch
Date: 2008-09-26 08:27:13
Message-ID: 48DC9CE1.3010707@kaltenbrunner.cc (view raw or flat)
Thread:
Lists: pgsql-hackers
Andrew Dunstan wrote:
> 
> Attached is my WIP patch for parallel pg_restore. It's still very rough, 
> but seems to work.
> 
> Anyone who can test this with highend equipment would be helping some.

tried playing with this(on a 22Gb compressed dump using 4 connections) 
but it does not seem to work at all for me:

pg_restore: [custom archiver] out of memory
pg_restore: [custom archiver] could not uncompress data: invalid block type
pg_restore: [custom archiver] out of memory
pg_restore: [custom archiver] could not uncompress data: invalid stored 
block lengths
pg_restore: [custom archiver] out of memory
pg_restore: [custom archiver] out of memory
pg_restore: [custom archiver] could not uncompress data: invalid 
distance too far back
pg_restore: [custom archiver] could not uncompress data: invalid 
distances set
pg_restore: [custom archiver] could not uncompress data: invalid code 
lengths set
pg_restore: [custom archiver] could not uncompress data: incorrect data 
check
pg_restore: [custom archiver] could not uncompress data: invalid code 
lengths set
pg_restore: [custom archiver] out of memory
pg_restore: [custom archiver] out of memory
pg_restore: [custom archiver] out of memory
pg_restore: [custom archiver] out of memory
pg_restore: [custom archiver] could not uncompress data: invalid 
literal/length code
pg_restore: [custom archiver] could not uncompress data: invalid 
literal/length code
pg_restore: [custom archiver] could not uncompress data: invalid block type

each pg_restore process seem to eat a few gigabytes of memory in a few 
seconds.


Stefan

In response to

Responses

pgsql-hackers by date

Next:From: Markus WannerDate: 2008-09-26 09:49:05
Subject: Re: Proposal: move column defaults into pg_attribute along with attacl
Previous:From: KaiGai KoheiDate: 2008-09-26 08:09:06
Subject: Re: Updates of SE-PostgreSQL 8.4devel patches

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group