Re: GUC_REPORT for protocol tunables was: Re: Optimize binary serialization format of arrays with fixed size elements

On Wed, Jan 25, 2012 at 01:43:03PM -0600, Merlin Moncure wrote:
> On Wed, Jan 25, 2012 at 1:24 PM, Marko Kreen <markokr(at)gmail(dot)com> wrote:
> > On Wed, Jan 25, 2012 at 12:54:00PM -0600, Merlin Moncure wrote:
> >> On Wed, Jan 25, 2012 at 11:24 AM, Marko Kreen <markokr(at)gmail(dot)com> wrote:
> >> > I specifically want to avoid any sort of per-connection
> >> > negotation, except the "max format version supported",
> >> > because it will mess up multiplexed usage of single connection.
> >> > Then they need to either disabled advanced formats completely,
> >> > or still do it per-query somehow (via GUCs?) which is mess.
> >>
> >> Being able to explicitly pick format version other than the one the
> >> application was specifically written against adds a lot of complexity
> >> and needs to be justified.  Maybe you're trying to translate data
> >> between two differently versioned servers?  I'm trying to understand
> >> the motive behind your wanting finer grained control of picking format
> >> version...
> >
> > You mean if client has written with version N formats, but connects
> > to server with version N-1 formats?  True, simply not supporting
> > such case simplifies client-side API.
> >
> > But note that it does not change anything on protocol level, it's purely
> > client-API specific.  It may well be that some higher-level APIs
> > (JDBC, Npgsql, Psycopg) may support such downgrade, but with lower-level
> > API-s (raw libpq), it may be optional whether the client wants to
> > support such usage or not.
>
> well, I see the following cases:
> 1) Vserver > Vapplication: server downgrades wire formats to
> applications version
> 2) Vapplication > Vlibpq > Vserver: since the application is
> reading/writing formats the server can't understand, an error should
> be raised if they are used in either direction
> 3) Vlibpq >= VApplication > Vserver: same as above, but libpq can
> 'upconvert' low version wire format to application's wire format or
> error otherwise.

I don't see why you special-case libpq here. There is no reason
libpq cannot pass older/newer formats through. Only thing that
matters it parser/formatter version. If that is done in libpq,
then app version does not matter. If it's done in app, then
libpq version does not matter.

> By far, the most common cause of problems (both in terms of severity
> and frequency) is case #1. #3 allows a 'compatibility mode' via
> libpq, but that comes at significant cost of complexity since libpq
> needs to be able to translate wire formats up (but not down). #2/3 is
> a less common problem though as it's more likely the application can
> be adjusted to get up to speed: so to keep things simple we can maybe
> just error out in those scenarios.

I don't like the idea of "conversion". Instead either client
writes values through API that picks format based on server version,
or it writes them for specific version only. In latter case it cannot
work with older server. Unless the fixed version is the baseline.

> In the database, we need to maintain outdated send/recv functions
> basically forever and as much as possible try and translate old wire
> format data to and from newer backend structures (maybe in very
> specific cases that will be impossible such that the application is
> SOL, but that should be rare). All send/recv functions, including
> user created ones need to be stamped with a version token (database
> version?). With the versions of the application, libpq, and all
> server functions, we can determine all wire formats as long as we
> assume the application's targeted database version represents all the
> wire formats it was using.
>
> My good ideas stop there: the exact mechanics of how the usable set of
> functions are determined, how exactly the adjusted type look ups will
> work, etc. would all have to be sorted out. Most of the nastier parts
> though (protocol changes notwithstanding) are not in libpq, but the
> server. There's just no quick fix on the client side I can see.

It does not need to be complex - just bring the version number to
i/o function and let it decide whether it cares about it or not.
Most functions will not.. Only those that we want to change in
compatible manner need to look at it.

But I don't see that there is danger of having regular changes in wire
formats. So most of the functions will ignore the versioning.
Including the ones that don't care about compatibility.

But seriously - on-wire compatibility is good thing, do not fear it...

--
marko