Re: Shared PostgreSQL libraries and symbol versioning

Greetings,

* Pavel Raiskup (praiskup(at)redhat(dot)com) wrote:
> On Thursday, May 10, 2018 12:41:40 PM CEST Pavel Raiskup wrote:
> > On Monday, April 9, 2018 11:04:33 PM CEST Tom Lane wrote:
> > > Stephen Frost <sfrost(at)snowman(dot)net> writes:
> > > > * Peter Eisentraut (peter(dot)eisentraut(at)2ndquadrant(dot)com) wrote:
> > > >> On 4/5/18 02:04, Pavel Raiskup wrote:
> > > >>> As a followup thought; there are probably two major obstacles ATM
> > > >>> - the DSOs' symbols are not yet versioned, and
> > > >>> - the build-system doesn't seem to know how to -lpq link against
> > > >>> external libpq.so
> > >
> > > > I've not looked but neither of those strike me as terribly difficult to
> > > > overcome, assuming they need to be overcome.
> > >
> > > I'm not excited about introducing YA cross-platform difference in our
> > > library/linking behavior without fairly strong evidence that it's
> > > necessary. The available evidence points in the other direction.
> >
> > Well, but I believe it is really needed (in our case at least) - it's so
> > important that we think about doing the symbol versioning downstream-only.
>
> Gently ping on this; sum-up is that we'll move libpq.so.5 into 'libpq'
> package (with corresponding libpq-devel having libpq.so). That package is
> likely to receive PG major version updates during one version of
> Fedora/CentOS/RHEL version; so because of that the plan is to start symbol
> versioning like RHPG_10@@..., e.g. 'PQinitOpenSSL@@RHPG_10'.
>
> I'd go with downstream change now, and I'd like to hear voices against this
> change (if there is anything obvious to you). I'd like to help having this
> as upstream opt-in of course, too, so feel free to ping me any time in future.

The way to try and move this forward would be to specifically address
Tom's concern about having more differences in our library/linking
behavior across platforms. Implementing this change in a way that makes
it very specific to one platform is surely going in the wrong direction.
Have you tried talking to the Debian and the maintainers of other
platforms to get their input on such a change? Are they supportive of
it, would they like to see upstream do it, what would make sense here if
upstream did it? Are there still platforms we care about which wouldn't
be able to support this? Is there a way to make it work across all
platforms?

The other side of this is: why isn't the packaging system handling this?
On Debian, at least, if you build a package with a newer version of a
library, then that package will depend on that newer version to be
installed; why isn't that enough for the RPM-based systems too? Older
packages will still run against the newer libpq unless/until we make a
backwards-incompatible change, which is rather rare.

> > As I said, we'd like to provide multiple major PG versions, but only one
> > set of PG libraries. But as the time will continue, supporting newer PG
> > major versions will require updating the system's default 'libpq.so.5',
> > potentially for the newly added symbols. If we had in our RPMs versioned
> > symbols, RPM would for free guard us against wrong package installations
> > (IOW RPM would guarantee that users won't install packages depending on
> > newer symbols unless also newer 'libpq.so.5' is installed). Without
> > lib symbol versions, there's no **easy** way around this.

I'm failing to see why symbol versioning is the way to prevent this from
happening.

> > Debian folks claim they don't have a problem with symbol versioning even
> > though they have the same installation pattern since ever, but IMO that's
> > because (as far as I understand how all of this is done on Debian):
> >
> > - Debian doesn't have that long support life cycle, so new symbols are
> > to occur less likely
> >
> > - Debian actually provides officially only one version of PG stack
> > (including libraries), the rest is provided through postgresql.org
> > repositories (one could say "third party", even though those are
> > probably the same maintainers). So for Debian, it's really unlikely to
> > build system package against updated 'libpq.so.5' which comes from
> > the "third party" repo.

I suspect you've actually missed the big point here that Debian has
versioned dependencies for libraries, provided by dpkg, and that they
track the new symbol additions and if an application is built against a
newer library which has new symbols then that application's package will
Depend on that newer version of the library. What it sounds like you're
saying here is that RPM will do that too- but only through the use of
symbol versioning.

> > I mean, worthless saying that Debian packaging would benefit from
> > versioned symbols too (== it would be safe to update system-wide package),
> > but it would be really awesome to have consistent (upstream blessed) way
> > to do the versioning.

On this point, I could possibly agree. Symbol versioning is something
I'm a fan of in general but we do need to think about the concern that
Tom raised, as well as consider if there's additional work that will
need to be done upstream for it. Debian played around with simply
slapping versioned symbols on whole libraries before and ran into some
trouble, as I recall, so I suspect there's more to be said here than
just "well, we'll just version everything every time." What happens
when a new release is cut and all of the symbols have been changed to be
newer- will all the existing packages have to be rebuilt to reference
those newer symbols and would all therefore depend on the newer libpq?
Or will we need to keep the symbol version for existing symbols the same
while managing newer symbols with newer symbol versions (ala what glibc
does, with no small amount of effort required to do so...).

> > Overall, what are your feelings? I'd be really glad to go through more
> > formal patch submission, those attachments are here just to have something more
> > concrete in hand for the discussion purposes.

Instead of just pushing on it, I'd suggest trying to address the concern
raised by Tom, and the other ones which I mention.

Thanks!

Stephen