Re: logical changeset generation v4 - Heikki's thoughts about the patch state

On Wed, Jan 23, 2013 at 5:30 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> pgbench upstream:
> tps: 22275.941409
> space overhead: 0%
> pgbench logical-submitted
> tps: 16274.603046
> space overhead: 2.1%
> pgbench logical-HEAD (will submit updated version tomorrow or so):
> tps: 20853.341551
> space overhead: 2.3%
> pgbench single plpgsql trigger (INSERT INTO log(data) VALUES(NEW::text))
> tps: 14101.349535
> space overhead: 369%
>
> Note that in the single trigger case nobody consumed the queue while the
> logical version streamed the changes out and stored them to disk.
>
> Adding a default NOW() or similar to the tables immediately makes
> logical decoding faster by a factor of about 3 in comparison to the
> above trivial trigger.
>
> The only reason the submitted version of logical decoding is
> comparatively slow is that its xmin update policy is braindamaged,
> working on that right now.

I agree. The thing that scares me about the logical replication stuff
is not that it might be slow (and if your numbers are to be believed,
it isn't), but that I suspect it's riddled with bugs and possibly some
questionable design decisions. If we commit it and release it, then
we're going to be stuck maintaining it for a very, very long time. If
it turns out to have serious bugs that can't be fixed without a new
major release, it's going to be a serious black eye for the project.

Of course, I have no evidence that that will happen. But it is a
really big piece of code, and therefore unless you are superman, it's
probably got a really large number of bugs. The scary thing is that
it is not as if we can say, well, this is a big hunk of code, but it
doesn't really touch the core of the system, so if it's broken, it'll
be broken itself, but it won't break anything else. Rather, this code
is deeply in bed with WAL, with MVCC, and with the on-disk format of
tuples, and makes fundamental changes to the first two of those. You
agreed with Tom that 9.2 is the buggiest release in recent memory, but
I think logical replication could easily be an order of magnitude
worse.

I also have serious concerns about checksums and foreign key locks.
Any single one of those three patches could really inflict
unprecedented damage on our community's reputation for stability and
reliability if they turn out to be seriously buggy, and unfortunately
I don't consider that an unlikely outcome. I don't know what to do
about it, either.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company