Re: CustomScan in a larger structure (RE: CustomScan support on readfuncs.c)

From: Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: CustomScan in a larger structure (RE: CustomScan support on readfuncs.c)
Date: 2015-11-13 04:36:42
Message-ID: 9A28C8860F777E439AA12E8AEA7694F80116FA14@BPXM15GP.gisp.nec.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> > On Wed, Nov 11, 2015 at 3:29 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> > > On 2015-11-11 14:59:33 -0500, Robert Haas wrote:
> > >> I don't see this as being a particularly good idea. The same issue
> > >> exists for FDWs, and we're just living with it in that case.
> > >
> > > It's absolutely horrible there. I don't see why that's a justification
> > > for much. To deal with the lack of extensible copy/out/readfuncs I've
> > > just had to copy the entirety of readfuncs.c into an extension. Or you
> > > build replacements for those (as e.g. postgres_fdw essentially has
> > > done).
> > >
> > >> If we do want to improve it, I'm not sure this is the way to go,
> > >> either. I think there could be other designs where we focus on making
> > >> the serialization and deserialization options better, rather than
> > >> letting people tack stuff onto the struct.
> > >
> > > Just better serialization doesn't actually help all that much. Being
> > > able to conveniently access data directly, i.e. as fields in a struct,
> > > makes code rather more readable. And in many cases more
> > > efficient. Having to serialize internal datastructures unconditionally,
> > > just so copyfuncs.c works if actually used, makes for a fair amount of
> > > inefficiency (forced deserialization, even when not copying) and uglier
> > > code.
> >
> > Fair enough, but I'm still not very interested in having something for
> > CustomScan only.
> >
> I agree with we have no reason why only custom-scan is allowed to have
> serialize/deserialize capability. I can implement an equivalent stuff
> for foreign-scan also, and it is helpful for extension authors, especially,
> who tries to implement remote join feature because FDW driver has to
> keep larger number of private fields than scan.
>
I tried to make a patch to support serialization/deserialization on both
of ForeignScan and CustomScan, but have not tested yet.

One exceptional stuff in ForeignScan is ForeignPath in outfuncs.c, because
ForeignPath itself does not have identifier to get callback functions (it
is kept in RelOptInfo; that is sufficient in planning phase), thus, we have
no way to write out if ForeignPath is a part of larger structure.
We ought to ignore it at this point. How about your opinion?

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

Attachment Content-Type Size
pgsql-v9.6-foreign-custom-serialization.v1.patch application/octet-stream 15.3 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2015-11-13 04:44:01 Re: Freeze avoidance of very large table.
Previous Message Kyotaro HORIGUCHI 2015-11-13 03:52:12 Re: Support for N synchronous standby servers - take 2