Re: [HACKERS] Custom compression methods

On Wed, Dec 13, 2017 at 7:18 AM, Ildus Kurbangaliev
<i(dot)kurbangaliev(at)postgrespro(dot)ru> wrote:
> Since we agreed on ALTER syntax, i want to clear things about CREATE.
> Should it be CREATE ACCESS METHOD .. TYPE СOMPRESSION or CREATE
> COMPRESSION METHOD? I like the access method approach, and it
> simplifies the code, but I'm just not sure a compression is an access
> method or not.

+1 for ACCESS METHOD.

> Current implementation
> ----------------------
>
> To avoid extra patches I also want to clear things about current
> implementation. Right now there are two tables, "pg_compression" and
> "pg_compression_opt". When compression method is linked to a column it
> creates a record in pg_compression_opt. This record's Oid is stored in
> the varlena. These Oids kept in first column so I can move them in
> pg_upgrade but in all other aspects they behave like usual Oids. Also
> it's easy to restore them.

pg_compression_opt -> pg_attr_compression, maybe.

> Compression options linked to a specific column. When tuple is
> moved between relations it will be decompressed.

Can we do this only if the compression method isn't OK for the new
column? For example, if the old column is COMPRESS foo PRESERVE bar
and the new column is COMPRESS bar PRESERVE foo, we don't need to
force decompression in any case.

> Also in current implementation SET COMPRESSION contains WITH syntax
> which is used to provide extra options to compression method.

Hmm, that's an alternative to use reloptions. Maybe that's fine.

> What could be changed
> ---------------------
>
> As Alvaro mentioned COMPRESSION METHOD is practically an access method,
> so it could be created as CREATE ACCESS METHOD .. TYPE COMPRESSION.
> This approach simplifies the patch and "pg_compression" table could be
> removed. So compression method is created with something like:
>
> CREATE ACCESS METHOD .. TYPE COMPRESSION HANDLER
> awesome_compression_handler;
>
> Syntax of SET COMPRESSION changes to SET COMPRESSION .. PRESERVE which
> is useful to control rewrites and for pg_upgrade to make dependencies
> between moved compression options and compression methods from pg_am
> table.
>
> Default compression is always pglz and if users want to change they run:
>
> ALTER COLUMN <col> SET COMPRESSION awesome PRESERVE pglz;
>
> Without PRESERVE it will rewrite the whole relation using new
> compression. Also the rewrite removes all unlisted compression options
> so their compresssion methods could be safely dropped.

That all sounds good.

> "pg_compression_opt" table could be renamed to "pg_compression", and
> compression options will be stored there.

See notes above.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company