Re: parallel restore vs. windows

From: Andrew Chernow <ac(at)esilo(dot)com>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Andrew Dunstan <andrew(at)dunslane(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: parallel restore vs. windows
Date: 2008-12-09 23:28:47
Message-ID: 493EFF2F.2030700@esilo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Magnus Hagander wrote:
> Tom Lane wrote:
>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>> Andrew Dunstan wrote:
>>>> I'll try. It's unfortunately not as simple as it sounds, because of the
>>>> way the abstractions are arranged. I can't count the number of times I
>>>> have had to stop and try to clear my head while working on this code.
>>> That's what killed me when I tried to review that stuff and figure it out.
>>> Does that indicate that the abstractions are bad and should be changed,
>>> or just that there's no reasonably way to make the abstractions both
>>> make sense for the internal API itself *and* for being threadsafe?
>> I think pretty much everybody except Philip Warner has found the stuff
>> around the TOC data structure and the "archiver" API to be confusing.
>> I'm not immediately sure about a better design though, at least not if
>> you don't want to duplicate a lot of code between the plain pg_dump and
>> the pg_dump/pg_restore cases.
>>
>> I don't see that this has much of anything to do with thread safety,
>> however --- it's just a matter of too many layers of indirection IMHO.
>
> It doesn't - but it makes it harder to find the issue I think :-( If it
> was reasonably easy, an API redesign might help that. But I haven't
> looked at all at the possibility of doing so, so I won't comment on if
> it's likely to be doable.
>
> //Magnus
>
>

If it previously worked without threads, than in theory a deep copy of the
thread_arg should fix the core dump; especially if the non-windows fork() method
works with this patch. Maybe you can get away with only copying some of the
members (trial-n-error), I don't think they are all being used in this context.
Nothing should be copied from within the thread itself.

--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Philip Warner 2008-12-10 00:19:58 Re: parallel restore vs. windows
Previous Message Andreas 'ads' Scherbaum 2008-12-09 23:16:28 FOSDEM 2009 Call for Papers - deadline