Re: parallel pg_restore - WIP patch

Stefan Kaltenbrunner wrote:
> Andrew Dunstan wrote:
>>
>> Attached is my WIP patch for parallel pg_restore. It's still very
>> rough, but seems to work.
>>
>> Anyone who can test this with highend equipment would be helping some.
>
> tried playing with this(on a 22Gb compressed dump using 4 connections)
> but it does not seem to work at all for me:
>
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] could not uncompress data: invalid block
> type
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] could not uncompress data: invalid
> stored block lengths
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] could not uncompress data: invalid
> distance too far back
> pg_restore: [custom archiver] could not uncompress data: invalid
> distances set
> pg_restore: [custom archiver] could not uncompress data: invalid code
> lengths set
> pg_restore: [custom archiver] could not uncompress data: incorrect
> data check
> pg_restore: [custom archiver] could not uncompress data: invalid code
> lengths set
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] could not uncompress data: invalid
> literal/length code
> pg_restore: [custom archiver] could not uncompress data: invalid
> literal/length code
> pg_restore: [custom archiver] could not uncompress data: invalid block
> type
>
> each pg_restore process seem to eat a few gigabytes of memory in a few
> seconds.

Ouch. Ok, Thanks for the report. I will investigate.

cheers

andrew