Re: Changeset Extraction v7.6.1

On 2014-02-17 21:10:26 -0500, Tom Lane wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> > 1. How safe is it to try to do decoding inside of a regular backend?
> > What we're doing here is entering a special mode where we forbid the
> > use of regular snapshots in favor of requiring the use of "decoding
> > snapshots", and forbid access to non-catalog relations. We then run
> > through the decoding process; and then exit back into regular mode.
> > On entering and on exiting this special mode, we
> > InvalidateSystemCaches().
>
> How often is such a mode switch expected to happen? I would expect
> frequent use of InvalidateSystemCaches() to be pretty much disastrous
> for performance, even absent any of the possible bugs you're worried
> about. It would likely be better to design things so that a decoder
> backend does only that.

Very infrequently. When it's starting to decode, and when it's
ending. When used via walsender, that should only happen at connection
start/end which surely shouldn't be frequent.
It's more frequent when using the SQL interface, but since that's not a
streaming interface on a busy server there still would be a couple of
megabytes of transactions to decode for one reset.

> > 3. As this feature is proposed, the only plugin we'll ship with 9.4 is
> > a test_decoding plugin which, as its own documentation says, "doesn't
> > do anything especially useful." What exactly do we gain by forcing
> > users who want to make use of these new capabilities to write C code?
>
> TBH, if that's all we're going to ship, I'm going to vote against
> committing this patch to 9.4 at all. Let it wait till 9.5 when we
> might be able to build something useful on it.

There *are* useful things around already. We didn't include postgres_fdw
in the same release as the fdw code either? I don't see why this should
be held to a different standard.

> To point out just
> one obvious problem, how much confidence can we have in the APIs
> being right if there are no usable clients?

Because there *are* clients. They just don't sound likely to either be
suitable for core code (to specialized) or have already been submitted
(the json plugin).

There's a whole replication suite built ontop of this, to a good degree
to just test it. So I am fairly confident that the most important parts
are covered. There sure is additional features I want, but that's not
surprising.

> The most recent precedent I can think of is the FDW APIs, which I'd
> be the first to admit are still in flux. But we didn't ship anything
> there without non-toy contrib modules to exercise it. If we had,
> we'd certainly have regretted it, because in the creation of those
> contrib modules we found flaws in the initial design.

Which non-toy fdw was there? file_fdw was in 9.1, but that's a toy. And
*8.4* had CREATE FOREIGN DATA WRAPPER, without it doing anything...

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services