Re: An idea for parallelizing COPY within one backend

From: "Tom Dunstan" <pgsql(at)tomd(dot)cc>
To: "Florian G(dot) Pflug" <fgp(at)phlo(dot)org>
Cc: "Brian Hurt" <bhurt(at)janestcapital(dot)com>, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Postgresql-Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: An idea for parallelizing COPY within one backend
Date: 2008-02-27 16:33:18
Message-ID: ca33c0a30802270833r54a8593egce1db9a082dc62a6@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Feb 27, 2008 at 9:26 PM, Florian G. Pflug <fgp(at)phlo(dot)org> wrote:
> I was thinking more along the line of letting a datatype specify a
> function "void* ioprepare(typmod)" which returns some opaque object
> specifying all that the input and output function needs to know.
> We could than establish the rule that input/output functions may not
> access the catalog, and instead pass them a pointer to that opaque object.

Callers of IO functions don't always know which type they're dealing
with - we had to go to some lengths to pass type information along
with the enum value itself so that it could be looked up in the
syscache in the output function. I think the main culprits are the
P/Ls, but I think there was a security related concern about passing
the type through to the IO function as well. If you want to do
something like this, it would certainly be possible to cache the enum
info for a particular type, but you might want to have a separate set
of io functions just for this case.

On the plus side, if such a cache were to be used by IO generally, we
could reimplement enums to just store the ordinal on disk and save a
couple of bytes, like I wanted but was unable to do the first time
around. :)

Enums are an easy case, though, as there's very little data to deal
with. I don't know about other UDTs out there - do any require more
extensive catalog access?

Cheers

Tom

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Hiroshi Saito 2008-02-27 16:34:03 Re: OSSP can be used in the windows environment now!
Previous Message Paul van den Bogaard 2008-02-27 15:58:57 Re: Proposed changes to DTrace probe implementation