Skip site navigation (1) Skip section navigation (2)

Re: pgsql_fdw, FDW for PostgreSQL server

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
Cc: Martijn van Oosterhout <kleptog(at)svana(dot)org>, Shigeru Hanada <shigeru(dot)hanada(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Kohei KaiGai <kaigai(at)kaigai(dot)gr(dot)jp>
Subject: Re: pgsql_fdw, FDW for PostgreSQL server
Date: 2011-10-29 21:42:36
Message-ID: 4819.1319924556@sss.pgh.pa.us (view raw or flat)
Thread:
Lists: pgsql-hackers
Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com> writes:
> On Sat, Oct 29, 2011 at 8:13 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I have not looked at the code, but ISTM the way that this has to work is
>> that you set up a portal for each active scan.  Then you can fetch a few
>> rows at a time from any one of them.

> Hmm, true. Looking back at the original proposal (neither did I look
> at the code,) there seems to be a cursor mode. ISTM it is hard for fdw
> to know how the whole plan tree looks, so consequently do we always
> cursor regardless of estimated row numbers?

I think we have to.  Even if we estimate that a given scan will return
only a few rows, what happens if we're wrong?  We don't want to blow out
memory on the local server by retrieving gigabytes in one go.

> I haven't had much experiences around cursor myself, but is it as
> efficient as non-cursor?

No, but if you need max efficiency you shouldn't be using foreign tables
in the first place; they're always going to be expensive to access.

It's likely that making use of native protocol portals (instead of
executing a lot of FETCH commands) would help.  But I think we'd be well
advised to do the first pass with just the existing libpq facilities,
and then measure to see where to improve performance.

			regards, tom lane

In response to

Responses

pgsql-hackers by date

Next:From: Jeff DavisDate: 2011-10-29 22:00:52
Subject: Re: strange code in array_in
Previous:From: Hitoshi HaradaDate: 2011-10-29 21:34:49
Subject: Re: pgsql_fdw, FDW for PostgreSQL server

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group