Re: parallel restore vs. windows

From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: Andrew Chernow <ac(at)esilo(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: parallel restore vs. windows
Date: 2008-12-09 19:45:39
Message-ID: 493ECAE3.6060600@hagander.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Andrew Dunstan wrote:
>
>
> Andrew Chernow wrote:
>>>>>
>>>>> Parts of AH need deep cloning, notably the formatData member, which
>>>>> is done in _ReopenArchive().
>>>>>
>>>>
>>>> Is it okay to clone this from within the thread?
>>>
>>> I don't see why not.
>>>
>>
>> Because another thread may be modifying the memory you are trying to
>> clone. If no one modifies the formatData struct, then why is it being
>> deep copied to begin with.
>>
>>>
>>> The program dies with a nasty dialog box when restoring a dump of the
>>> regression database after the second COPY thread disconnects.
>>
>> Sounds like the friendly and helpful GPF Dialog (General Protection
>> Fault). This is a core dump which strongly suggests your threads are
>> trampling over one another. Its possible that a couple threads get
>> fired off but upon the first thread completion, something
>> !(deep_copied) is freed/modified ... bang-bang your dead :o I tried
>> to find this, but haven't yet.
>>
>> Maybe do a full deep copy in the main thread and comment out any
>> in-thread deep copying. I wonder if that would work with no other
>> changes.
>
> I'll try. It's unfortunately not as simple as it sounds, because of the
> way the abstractions are arranged. I can't count the number of times I
> have had to stop and try to clear my head while working on this code.

That's what killed me when I tried to review that stuff and figure it out.

Does that indicate that the abstractions are bad and should be changed,
or just that there's no reasonably way to make the abstractions both
make sense for the internal API itself *and* for being threadsafe?

//Magnus

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2008-12-09 19:52:52 Re: SSL BIO wrappers
Previous Message Magnus Hagander 2008-12-09 19:44:15 Re: SSL BIO wrappers