Re: [HACKERS] Custom compression methods

On Thu, Mar 11, 2021 at 10:07 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby <pryzby(at)telsasoft(dot)com> wrote:
> For INSERT SELECT we were already doing in the older version so we can
> include that code here, we will also have to include the patches for
> decompressing data before forming the composite types because without
> that we can not ensure that lz4 does not exist anywhere in the table.
> Said that with that also we can not ensure that it doesn't exist anywhere
> in the system because it might exist in the WAL and if you do the crash
> recovery then might get those lz4 compressed data back.

In updated patches, now INSERT INTO SELECT/VACUUM FULL/ CLUSTER will
re compress the data as per the latest compression method.

create table t(a text compression lz4);
insert into t values(repeat('a', 9000));
postgres[129360]=# select pg_column_compression(a) from t;
pg_column_compression
-----------------------
lz4
(1 row)

postgres[129360]=# alter TABLE t ALTER COLUMN a SET COMPRESSION pglz;
ALTER TABLE
postgres[129360]=# select pg_column_compression(a) from t;
pg_column_compression
-----------------------
lz4
(1 row)

postgres[129360]=# VACUUM FULL t;
VACUUM
postgres[129360]=# select pg_column_compression(a) from t;
pg_column_compression
-----------------------
pglz
(1 row)

IMHO, now we have a way for user to rewrite table using VACUUM
FULL/CLUSTER so I don't think we should force rewrite in ALTER SET
COMPRESSION.

In attached patch I am re compressing in INSERT INTO SELECT as well,
but honestly I think maybe we don't need to force that also on user
and we can only do this in VACUUM FULL or CLUSTER, thoughts?

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com