Re: bg worker: patch 1 of 6 - permanent process

On Thu, Sep 16, 2010 at 4:47 AM, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> <showing-off>
> BTW, that'd be what I call a huge patch:
>
> bgworkers, excluding dynshmem and imessages:
>  34 files changed, 2910 insertions(+), 1421 deletions(-)
>
> from there to Postgres-R:
>  98 files changed, 14856 insertions(+), 230 deletions(-)
> </showing-off>

Yeah, that's huge. :-)

>> That
>> conversation probably needs to start from the other end - is the
>> overall architecture correct for us? - before we get down to specific
>> patches.  On the other hand, I'm very interested in laying the
>> groundwork for parallel query
>
> Cool. Maybe we should take another look at bgworkers, as soon as a parallel
> querying feature gets planned?

Well, that will obviously depend somewhat on the wishes of whoever
decides to work on parallel query, but it seems reasonable to me. It
would be nice to get some pieces of this committed incrementally but
as I say I fear there is too much dependency on what might happen
later, at least the way things are structured now.

>> and I think there are probably a number
>> of bits of architecture both from this project and Postgres-XC, that
>> could be valuable contributions to PostgreSQL;
>
> (...note that Postgres-R is license compatible, as opposed to the GPL'ed
> Postgres-XC project...)

Yeah. +1 for license compatibility.

> As you can certainly imagine, it's important for me that any modification to
> such a patch from Postgres-R would still be compatible to what I use it for
> in Postgres-R and not cripple any functionality there, because that'd
> probably create more work for me than not getting the patch accepted
> upstream at all.

That's an understandable goal, but it may be difficult to achieve.

> As long as you don't consider imessages and dynshmem a part of Postgres-R,
> they are independent of the rest of Postgres-R in the technical sense.
>
> And for any kind of parallel querying feature, imessages and dynshmem might
> be of help as well. So I currently don't see where I could de-couple these
> patches any further.

I agree. I've already said my piece on how I think that stuff would
need to be reworked to be acceptable, so we might have to agree to
disagree on those, especially if your goal is to get something
committed that doesn't involve a major rewrite on your end.

> I didn't really expect them to get accepted to Postgres core at the moment.
> But the Postgres team normally asks for sharing concepts and ideas as early
> as possible...

Absolutely.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company