Re: [HACKERS] Custom compression methods

From: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Andres Freund <andres(at)anarazel(dot)de>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>, David Steele <david(at)pgmasters(dot)net>, Ildus Kurbangaliev <i(dot)kurbangaliev(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [HACKERS] Custom compression methods
Date: 2021-03-17 11:40:44
Message-ID: CAFiTN-vhMut4QBrMNBks9=L5ug5hDV3AXA9jkLv+juebF-sRzg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Mar 17, 2021 at 1:01 AM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
Please find the updated version of the patch set.
Changes:
0001:
- Added comment for PG_RETURN_HEAPTUPLEHEADER_RAW().
- Added assert in heap_copy_tuple_as_raw_datum() that there should be
no external data in tuple

0002:
- Wrapper over heap_form_tuple and used in ExecEvalRow() and
ExecEvalFieldStoreForm()

0003:
1. Used 2 bits of varatt_external.va_extsize to store compression method
2. Removed logic of compressing.
3. Some other minor cleanup

0004: some minor cleanup
0005: Removed code of recompress from ATRewriteTuple, now only
recompress happens in Vacuum FULL and CLUSTER. So that there is at
least some way to recompress. Also, here we don't need to take the
overhead of deforming the tuple so it should not affect performance
also in unrelated cases. We will have to pay the penalty if we need
to decompress a lot of attributes.
0006: removed as suggested by Robert
0007: now 0006, only for CFbot

-- I still have the HIDE_TOAST_COMPRESSION code in the patch set do we
think we need to remove? I don't mind keeping it as it is because we
can avoid changes in unrelated regression tests.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Attachment Content-Type Size
v36-0002-Expand-the-external-data-before-forming-the-tupl.patch text/x-patch 8.2 KB
v36-0004-Add-default_toast_compression-GUC.patch text/x-patch 11.5 KB
v36-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch text/x-patch 37.4 KB
v36-0003-Built-in-compression-method.patch text/x-patch 105.0 KB
v36-0005-Alter-table-set-compression.patch text/x-patch 21.4 KB
v36-0006-default-to-with-lz4.patch text/x-patch 1.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Euler Taveira 2021-03-17 11:50:26 Re: a verbose option for autovacuum
Previous Message Bharath Rupireddy 2021-03-17 11:22:34 Re: logical replication worker accesses catalogs in error context callback