Re: The plan for FDW-based sharding

On Fri, Mar 11, 2016 at 9:09 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:

>
>
>
> 3. I have tried to encourage others to get involved, with limited
> success. I do think the FDW is perhaps the only reasonable way to get
> _built-in_ sharding. The external sharding solutions are certainly
> viable, but external. It is possible we will make all the FDW
> improvements, find out it doesn't work, but find out the improvements
> allow us to go in another direction.
>

I remember last summer emails and we really wanted to participate in
development, but it happens all slots were occupied by edb and ntt people.
We wanted to work on distributed transactions and proposed our XTM. Our
feeling that time from discussion was that we were invited, but all doors
were closed. It was very bad experience. Hopefully, we understand our
misunderstanding.

>
> There seems to be serious interest in how this idea came about, so let
> me say what I remember.
>

I think the idea was so obvious, so let's don't discuss this.

>
> As for why there is so much hostility, I think this is typical for any
> ill-defined feature development. There was simmering hostility to the
> Windows port and pg_upgrade for many years because those projects were
> not easy to define and risky, and had few active developers. The
> agreement was that work could continue as long as destabilization wasn't
> introduced. Ideally everything would have a well-defined plan, it is
> sometimes hard to do. Similar to our approach on parallelism (which is
> also super-important and doesn't many active developers), sometimes you
> just need to create infrastructure and see how well it solves problems.
>
>

Our XTM is the yet another example of infrastructure we need to work on
clustering. Should we wait other smart guy starts thinking on distributed
transactions ? We described in https://wiki.postgresql.org/wiki/DTM our
API, which is just a wrapper on existed functions, but it will allow us and
fortunately others to play with their ideas. We did several prototypes,
including FDW, to demonstrate viability of API, and plan to continue our
work on built-in high availability, multi-master. Of course, there will be
a lot to learn, but it will be much easier if XTM will exists not as
separate patch, which is really small.

>
> --
> Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
> EnterpriseDB http://enterprisedb.com
>
> + As you are, so once was I. As I am, so you will be. +
> + Roman grave inscription +
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>