parallel restore

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: parallel restore
Date: 2009-01-09 14:34:14
Message-ID: 49676066.9070909@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Attached is the latest version.

Changes:

. some tidy up as variously requested.
. some common code is factored out
. some descriptive comments added
. platform specific stuff (e.g. spawn, reap) is factored out
. --truncate_before_load is gone, and we now do this during parallel
restore if we created the table. For a non-parallel restore the
equivalent would be to run the whole restore in a single transaction.

One of the hardest parts of getting this to work was handling
dependencies right. I will work on adding some more comments regarding that.

Simon asked about a way to adjust the number of worker children as we go
along. That's way out of scope at this stage. In testing it appears that
the sweet spot is roughly m=number_of_processors, which makes some
sense, but more experience will clarify this.

cheers

andrew

Attachment Content-Type Size
parallel_restore_14.patch.gz application/x-gzip 17.7 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2009-01-09 14:39:49 Re: SET TRANSACTION and SQL Standard
Previous Message Bruce Momjian 2009-01-09 14:31:32 Re: Improving compressibility of WAL files