Re: [HACKERS] Custom compression methods

From: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
To: Justin Pryzby <pryzby(at)telsasoft(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>, David Steele <david(at)pgmasters(dot)net>, Ildus Kurbangaliev <i(dot)kurbangaliev(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [HACKERS] Custom compression methods
Date: 2021-03-15 12:14:33
Message-ID: CAFiTN-s8SsWs2zCZL3TFxXwi4VE35J=4aKpduKVrTaL+3wnB0A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Mar 12, 2021 at 4:06 PM Justin Pryzby <pryzby(at)telsasoft(dot)com> wrote:
>
> I think these names need to be more specific.
>
> +typedef enum CompressionId
> +{
> + PGLZ_COMPRESSION_ID = 0,
> + LZ4_COMPRESSION_ID = 1
> +
>
> CompressionId, PGLZ_COMPRESSION_ID, LZ4_COMPRESSION_ID are also being used by
> Andrey's WAL compression patch. I suggested he use a prefix, but your patch is
> also of limited scope (TOAST).

In the attached patches I have changed this, other than this I have
run pg_indent on all the patches and now CTAS and Matview will compare
the compression method with the target attribute and it will
recompress if required. CLUSTER, VACUUM
FULL(reform_and_rewrite_tuple) and ATRewriteTable will also recompress
as per the target attribute. But the changes of
reform_and_rewrite_tuple and ATRewriteTable are in patch 0005, because
now whenever we are moving to target table we are always recompress so
there is no use-case to recompress in reform_and_rewrite_tuple and
ATRewriteTable unless we provide an option to alter the attribute
compression. Basically, we are allowed to change the attribute
compression without any rewrite but on the very next opportunity when
the table is getting rewritten the data will be recompressed based on
the current compression method. Also done some improvement and
comment changes in patch 0002 and patch 0006.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Attachment Content-Type Size
v35-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch text/x-patch 37.0 KB
v35-0002-Expand-the-external-data-before-forming-the-tupl.patch text/x-patch 6.7 KB
v35-0004-Add-default_toast_compression-GUC.patch text/x-patch 11.5 KB
v35-0005-Alter-table-set-compression.patch text/x-patch 25.3 KB
v35-0003-Built-in-compression-method.patch text/x-patch 108.8 KB
v35-0006-Disallow-compressed-data-inside-container-types.patch text/x-patch 11.9 KB
v35-0007-default-to-with-lz4.patch text/x-patch 1.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amul Sul 2021-03-15 12:15:22 Re: [Patch] ALTER SYSTEM READ ONLY
Previous Message Matthias van de Meent 2021-03-15 11:43:40 Re: Improvements and additions to COPY progress reporting