Re: Future In-Core Replication

On Mon, Apr 30, 2012 at 6:43 PM, Josh Berkus <josh(at)agliodbs(dot)com> wrote:
> Well, this *is* the purpose of the cluster-hackers group, to add backend
> support which would make external replication systems easier to build
> and more efficient.  So far the only real feature to come out of that
> has been the Command Triggers, but if you read the TODO list of that
> group you'll see that it's a laundry list of things replication systems
> need support for in the backend.

Right, but the cluster-hackers group has essentially zero mailing list
traffic and seems to be making no progress on getting any of the
features they need into the backend, with the exception, as you say,
of Command Triggers. So to say, oh, well, we don't need a new effort
to do this, we have cluster-hackers is just sticking our head in the
sand. We obviously do need more of an effort than we've heretofore
had, and if Simon/2Q are willing to step it up a few notches, kudos to
them. Hopefully I'll get a chance to participate as well.

One thing I am a bit uncomfortable with about this whole discussion is
that it seems like a lot of the design and research is happening
off-list, with intent to report results back here later. While that
is obviously fine at some level, keeping the design under wraps until
the code is written is a strategy that's had notoriously poor success
in this community, and I hope we're not in for a repeat of that
strategy. I think we ought to be sharing and debugging designs in
public, not internally within 2ndQuadrant - or any other company, or
any other mailing list other than this one. Nobody enjoys doing a lot
of work on a patch and then having it get killed because Tom Lane - or
someone else - points out some critical flaw in the design, and the
way to avoid that is to try to flush out the flaws in public before
too much work goes into it. On the flip side I certainly understand
that sometimes you need to take some time to get your house in order
before you can present a coherent public proposal, so don't take an
accusation that anyone is doing anything wrong, just a possible
concern.

> What puts me off about this "let's start from first principles" approach
> is that in our community we have years of experience (in a couple cases,
> over a decade) with logical-level replication.  It seems like the last
> thing we need is Yet Another PostgreSQL Replication System, started over
> from scratch and years away from being production quality.  Don't we
> have enough external replication systems with not enough developers
> behind them?

No. We have, after over a decade of work, no meaningful core support
for the parts of replication that need core support in order to not
suck. People are switching from logical replication of various types
to SR in droves, not because they like the draconian restrictions that
HS/SR impose, but rather because it has superior performance and
reliability characteristics. It has those characteristics because it
has core support. You cannot build an airplane by bolting wings onto
a car.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company