Re: [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

On Wednesday, June 13, 2012 08:50:42 PM Christopher Browne wrote:
> On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund <andres(at)2ndquadrant(dot)com>
wrote:
> > From: Andres Freund <andres(at)anarazel(dot)de>
> >
> > We decided to use low level functions to do the apply instead of
> > producing sql statements containing the data (or using prepared
> > statements) because both, the text conversion and the full executor
> > overhead aren't introduce a significant overhead which is unneccesary if
> > youre using the same version of pg on the same architecture.
> >
> > There are loads of use cases though that require different methods of
> > applyin though - so the part doing the applying from an ApplyCache is
> > just a bunch of well abstracted callbacks getting passed all the
> > required knowledge to change the data representation into other formats.
>
> It's important to make sure that it's not going to be *too* difficult
> to "jump through the hoops" necessary to apply changes on a different
> version.
I aggree. But I don't see it as a feature of the first version for the moment.
Getting a base set of features into 9.3 is going to be hard enough as-is. But
I think there is enough interest from all sides to make that possible cross-
version.

> While pg_upgrade has diminished the need to use replication to handle
> cross-version/architecture upgrades, I don't think it has brought that
> to zero.
Aggreed.

> One other complication I'll observe... The assumption is being made
> that UPDATE/DELETE will be handled via The Primary Key. For the most
> part, I approve of this. Once upon a time, Slony had a misfeature
> where you could tell it to add in a surrogate primary key, and that
> caused no end of trouble. However, the alternative, that *does* seem
> to work alright, is to allow selecting a candidate primary key, that
> is, a set of columns that have UNIQUE + NOT NULL characteristics. I
> could see people having a problem switching over to use this system if
> they MUST begin with a 'Right Proper Primary Key.' If they start with
> a system with a 2TB table full of data that lacks that particular
> constraint, that could render them unable to use the facility.
It wouldn't need that much code to allow candidate keys. The data
representation in the catalogs is a bit unfriendly for that, but there has
been talk about changing that for some time now. I am not convinced that its
worth the cpu cycles though.

Btw, you can convert a unique key to a primary key since 9.1. The unique key
previously can be created CONCURRENTLY.

> > Missing:
> >
> > - TOAST handling. For physical apply not much needs to be done because
> > the toast inserts will have been made beforehand. There needs to be an
> > option in ApplyCache that helps reassembling TOAST datums to make it
> > easier to write apply modules which convert to text.
> Dumb question: Is it possible that two nodes would have a different
> idea as to which columns need to get toasted? I should think it
> possible for nodes to be configured with different values for TOAST
> policies, and while it's likely pretty dumb to set them to have
> different handlings, it would seem undesirable to not bother looking,
> and find the backend crashing due to an un-noticed mismatch.
I don't think it should be possible to configure the toast configurations
differently if you use the "binary apply" mode. But even if it were a value
which is toasted although the local policy says it should not be wouldn't
cause any problems as far as I can see.
The one problem that could cause problems for that are different page sizes et
al, but that needs to be prohibited anyway.

Andres

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services