Concurrent Restores

From: Volkan YAZICI <yazicivo(at)ttmail(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Concurrent Restores
Date: 2008-07-03 08:10:36
Message-ID: 87abgzpev7.fsf@alamut.mobiliz.com.tr
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

[I've searched archives for the subject, but couldn't find a related
discussion. If there is any, sorry for duplication.]

We're migrating nearly a dozen of MSSQL servers of size ~100GiB per
cluster. For this purpose, we dump MSSQL data to COPY files using a Java
program. We have database schemas for PostgreSQL which are equivalent to
their correponding ones in MSSQL side. The problem is, while we're
creating primary key, foreign key and index relations, I'm manually
partitioning related SQL files into separate files to gain performance
from CPU usage. One can argue that, concurrent processes will consume
larger disk I/O in this scheme and cause I/O bottleneck this time. But
as far as I monitored the system statistics, during concurrent
restoration, in our situation operation is CPU bounded, not disk
I/O. (Thanks SAN!)

pg_dump is capable of dumping objects with respect to their dependency
relations. It'd be really awesome if pg_dump can also handle
parallelizing primary key, foreign key and index creation queries into
separate files. Would such a think be possible? Comments?

Regards.

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Marko Kreen 2008-07-03 08:15:10 Re: [PATCHES] pg_dump lock timeout
Previous Message Teodor Sigaev 2008-07-03 07:31:06 Re: PATCH: CITEXT 2.0