Re: [HACKERS] Custom compression methods

From: Justin Pryzby <pryzby(at)telsasoft(dot)com>
To: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>, David Steele <david(at)pgmasters(dot)net>, Ildus Kurbangaliev <i(dot)kurbangaliev(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: [HACKERS] Custom compression methods
Date: 2021-03-12 03:24:09
Message-ID: 20210312032409.GJ8796@telsasoft.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Mar 01, 2021 at 08:53:09PM +0530, Dilip Kumar wrote:
> On Mon, Mar 1, 2021 at 5:36 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> > On Mon, Mar 1, 2021 at 11:06 AM Justin Pryzby <pryzby(at)telsasoft(dot)com> wrote:
> > > Thanks. It seems like that explains it.
> > > I think if that's a problem with recent versions, then you'll have to
> > > conditionally disable slicing.
> > > https://packages.debian.org/liblz4-dev
> > >
> > > Slicing isn't generally usable if it sometimes makes people's data inaccessible
> > > and gives errors about corruption.
> > >
> > > I guess you could make it a compile time test on these constants (I don't know
> > > the necessary version, though)
> > >
> > > #define LZ4_VERSION_MAJOR 1 /* for breaking interface changes */
> > > #define LZ4_VERSION_MINOR 7 /* for new (non-breaking) interface capabilities */
> > > #define LZ4_VERSION_RELEASE 1 /* for tweaks, bug-fixes, or development */
> > > #define LZ4_VERSION_NUMBER (LZ4_VERSION_MAJOR *100*100 + LZ4_VERSION_MINOR *100 + LZ4_VERSION_RELEASE)
> > >
> > > If the version is too low, either make it #error, or disable slicing.
> > > The OS usual library version infrastructure will make sure the runtime version
> > > is at least the MAJOR+MINOR of the compile time version.
> >
> > I think we can check the version and if it too low i.e. below1.8.3 (
> > in this release the slicing issue was fixed) then we can call the full
> > decompression routine from the slicing function.

Thank you

+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else

It occurred to me that this should actually compare the runtime version with
LZ4_versionNumber(). That way, a library upgrade can enable the slice
behavior.

--
Justin

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2021-03-12 03:31:28 Re: Key management with tests
Previous Message Andres Freund 2021-03-12 03:22:57 Re: shared-memory based stats collector