From: | Itagaki Takahiro <itagaki(dot)takahiro(at)gmail(dot)com> |
---|---|
To: | Joachim Wieland <joe(at)mcknight(dot)de> |
Cc: | Robert Haas <robertmhaas(at)gmail(dot)com>, Jaime Casanova <jaime(at)2ndquadrant(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: pg_dump directory archive format / parallel pg_dump |
Date: | 2011-02-04 04:46:12 |
Message-ID: | AANLkTin2ce85MeFLT558rrSCVjTDATDuVJrf1XJA-2+2@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Wed, Feb 2, 2011 at 13:32, Joachim Wieland <joe(at)mcknight(dot)de> wrote:
> Here is a rebased version with some minor changes as well.
I read the patch works as below. Am I understanding correctly?
1. Open all connections in a parent process.
2. Start transactions for each connection in the parent.
3. Spawn child processes with fork().
4. Each child process uses one of the inherited connections.
I think we have 2 important technical issues here:
* The consistency is not perfect. Each transaction is started
with small delays in step 1, but we cannot guarantee no other
transaction between them.
* Can we inherit connections to child processes with fork() ?
Moreover, we also need to pass running transactions to children.
I wonder libpq is designed for such usage.
To solve both issues, we might want a way to control visibility
in a database server instead of client programs. Don't we need
server-side support like [1] before developing parallel dump?
[1] http://wiki.postgresql.org/wiki/ClusterFeatures#Export_snapshots_to_other_sessions
> I haven't
> tested it on Windows now but will do so as soon as the Unix part has
> been reviewed.
It might be better to remove Windows-specific codes from the first try.
I doubt Windows message queue is the best API in such console-based
application. I hope we could use the same implementation for all
platforms for inter-process/thread communication.
--
Itagaki Takahiro
From | Date | Subject | |
---|---|---|---|
Next Message | David Wilson | 2011-02-04 05:06:18 | Re: [HACKERS] Slow count(*) again... |
Previous Message | Robert Haas | 2011-02-04 04:42:22 | Re: Compilation failed |