"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> How exactly are you allocating tasks to threads in this prototype,
> Right there is no balance here. Let me explain what I did. I performed
> a pg_restore -l to get the TOC file. I then broke that file up into
> five other files.
> prefix = schema (minus indexes, constraints)
> data = data
> pk = primary keys
> index = indexes
> triggers_constraints = well triggers and cosntraints (foreign keys in
> this instance)
> The first step of the script loads prefix. It then splits the data
> file into -n- number of files and launches -n- number of
> pg_restore processes with -L.
> It runs through all data, then starts on pk in the exact same manner
> and then indxex etc...
So you have four serialization points not just one; at each one the
slowest subtask forces everyone else to wait, even if there's work that
could potentially be done on other tables. This is fine for a
quick-and-dirty proof of concept but it's certainly not how we'd want to
implement the real thing. But I doubt you can get much further without
putting some actual dependency awareness into it.
regards, tom lane
In response to
pgsql-hackers by date
|Next:||From: Joshua D. Drake||Date: 2008-02-27 00:24:01|
|Subject: Re: multi-worker pg_restore was: 8.3 / 8.2.6 restore
|Previous:||From: Tom Lane||Date: 2008-02-26 23:59:16|
|Subject: Re: Required make version |