Skip site navigation (1) Skip section navigation (2)

Re: parallel pg_restore - WIP patch

From: Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: Russell Smith <mr-russ(at)pws(dot)com(dot)au>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Jeffrey Baker <jwbaker(at)gmail(dot)com>
Subject: Re: parallel pg_restore - WIP patch
Date: 2008-09-29 11:55:03
Message-ID: 48E0C217.4070007@kaltenbrunner.cc (view raw or flat)
Thread:
Lists: pgsql-hackers
Andrew Dunstan wrote:
> 
> 
> Andrew Dunstan wrote:
>>
>>
>>>
>>> this works better but there is something fishy still - using the same 
>>> dump file I get a proper restore using pg_restore normally. If I 
>>> however use -m for a parallel one I only get parts (in this case only 
>>> 243 of the 709 tables) of the database restored ...
>>>
>>>
>>>
>>
>> Yes, there are several funny things going on, including some stuff 
>> with dependencies. I'll have a new patch tomorrow with luck. Thanks 
>> for testing.
>>
>>
> 
> OK, in this version a whole heap of bugs are fixed, mainly those to do 
> with dependencies and saved state. I get identical row counts in the 
> source and destination now, quite reliably.

this looks much better (for a restore that usually takes 180min I can 
get down to 72min using -m 4) - however especially with higher 
concurrency I'm sometimes running into restore failures due to deadlocks 
happening during constraint restoration (slightly redacted):

pg_restore: [archiver (db)] Error from TOC entry 7765; 2606 1460743180 
FK CONSTRAINT fk_av_relations_av db_owner
pg_restore: [archiver (db)] could not execute query: ERROR:  deadlock 
detected
DETAIL:  Process 18100 waits for AccessExclusiveLock on relation 
1460818342 of database 1460815284; blocked by process 18103.
Process 18103 waits for AccessExclusiveLock on relation 1460818336 of 
database 1460815284; blocked by process 18100.
HINT:  See server log for query details.

ALTER TABLE ONLY foo
     ADD CONSTRAINT fk_av_relations_av FOREIGN KEY (vs_id) REFERENCES 
bar ...

In response to

Responses

pgsql-hackers by date

Next:From: Tom LaneDate: 2008-09-29 12:06:23
Subject: Re: parallel pg_restore - WIP patch
Previous:From: pgsqlDate: 2008-09-29 11:16:30
Subject: Re: Ad-hoc table type?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group