Re: [HACKERS] Custom compression methods

From: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
To: Justin Pryzby <pryzby(at)telsasoft(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>, David Steele <david(at)pgmasters(dot)net>, Ildus Kurbangaliev <i(dot)kurbangaliev(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [HACKERS] Custom compression methods
Date: 2021-03-01 12:06:55
Message-ID: CAFiTN-vp8RTR0m-50Qi5cq-+PE9aiTxBwRXTd0o_Hea7gm0meg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Mar 1, 2021 at 11:06 AM Justin Pryzby <pryzby(at)telsasoft(dot)com> wrote:

> Thanks. It seems like that explains it.
> I think if that's a problem with recent versions, then you'll have to
> conditionally disable slicing.
> https://packages.debian.org/liblz4-dev
>
> Slicing isn't generally usable if it sometimes makes people's data inaccessible
> and gives errors about corruption.
>
> I guess you could make it a compile time test on these constants (I don't know
> the necessary version, though)
>
> #define LZ4_VERSION_MAJOR 1 /* for breaking interface changes */
> #define LZ4_VERSION_MINOR 7 /* for new (non-breaking) interface capabilities */
> #define LZ4_VERSION_RELEASE 1 /* for tweaks, bug-fixes, or development */
> #define LZ4_VERSION_NUMBER (LZ4_VERSION_MAJOR *100*100 + LZ4_VERSION_MINOR *100 + LZ4_VERSION_RELEASE)
>
> If the version is too low, either make it #error, or disable slicing.
> The OS usual library version infrastructure will make sure the runtime version
> is at least the MAJOR+MINOR of the compile time version.

I think we can check the version and if it too low i.e. below1.8.3 (
in this release the slicing issue was fixed) then we can call the full
decompression routine from the slicing function.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message luis.roberto 2021-03-01 12:49:47 Re: Improving connection scalability: GetSnapshotData()
Previous Message Amit Kapila 2021-03-01 12:02:06 Re: [PATCH] Note effect of max_replication_slots on subscriber side in documentation.