From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Dilip Kumar <dilipbalaut(at)gmail(dot)com> |
Cc: | Justin Pryzby <pryzby(at)telsasoft(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>, David Steele <david(at)pgmasters(dot)net>, Ildus Kurbangaliev <i(dot)kurbangaliev(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: [HACKERS] Custom compression methods (mac+lz4.h) |
Date: | 2021-03-22 14:41:48 |
Message-ID: | 558766.1616424108@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Dilip Kumar <dilipbalaut(at)gmail(dot)com> writes:
> On Mon, Mar 22, 2021 at 5:22 AM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Also, after studying the documentation for LZ4_decompress_safe
>> and LZ4_decompress_safe_partial, I realized that liblz4 is also
>> counting on the *output* buffer size to not be a lie. So we
>> cannot pass it a number larger than the chunk's true decompressed
>> size. The attached patch resolves the issue I'm seeing.
> Okay, the fix makes sense. In fact, IMHO, in general also this fix
> looks like an optimization, I mean when slicelength >=
> VARRAWSIZE_4B_C(value), then why do we need to allocate extra memory
> even in the case of pglz. So shall we put this check directly in
> toast_decompress_datum_slice instead of handling it at the lz4 level?
Yeah, I thought about that too, but do we want to assume that
VARRAWSIZE_4B_C is the correct way to get the decompressed size
for all compression methods?
(If so, I think it would be better style to have a less opaque macro
name for the purpose.)
regards, tom lane
From | Date | Subject | |
---|---|---|---|
Next Message | Justin Pryzby | 2021-03-22 14:44:47 | Re: [HACKERS] Custom compression methods |
Previous Message | Robert Haas | 2021-03-22 14:41:33 | Re: [HACKERS] Custom compression methods |