Re: An idea for parallelizing COPY within one backend

From: Brian Hurt <bhurt(at)janestcapital(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Florian G(dot) Pflug" <fgp(at)phlo(dot)org>, Postgresql-Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: An idea for parallelizing COPY within one backend
Date: 2008-02-27 15:05:38
Message-ID: 47C57C42.5000305@janestcapital.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tom Lane wrote:

>"Florian G. Pflug" <fgp(at)phlo(dot)org> writes:
>
>
>>...
>>Neither the "dealer", nor the "workers" would need access to the either
>>the shared memory or the disk, thereby not messing with the "one backend
>>is one transaction is one session" dogma.
>>...
>>
>>
>
>Unfortunately, this idea has far too narrow a view of what a datatype
>input function might do. Just for starters, consider "enum" input,
>which certainly requires catalog access. We have also explicitly
>acknowledged the idea that datatype I/O functions might try to store
>typmod-related data in some special catalog somewhere.
>
> regards, tom lane
>
>
>
Would it be possible to determine when the copy is starting that this
case holds, and not use the parallel parsing idea in those cases?

I'm a big user of copy, generally into very simple tables- few indexes,
simple column types (numeric, varchar, and int almost exclusively), no
fancy features. A parallel copy input in the "simple" cases would be of
great advantage to me, even if it doesn't parallelize "complicated" cases.

Brian

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2008-02-27 15:10:02 Re: An idea for parallelizing COPY within one backend
Previous Message A.M. 2008-02-27 14:35:27 Re: An idea for parallelizing COPY within one backend