Re: [HACKERS] Custom compression methods

From: Ildus Kurbangaliev <i(dot)kurbangaliev(at)postgrespro(dot)ru>
To: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Andres Freund <andres(at)anarazel(dot)de>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Oleg Bartunov <obartunov(at)gmail(dot)com>, Craig Ringer <craig(at)2ndquadrant(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [HACKERS] Custom compression methods
Date: 2017-11-20 09:44:28
Message-ID: 20171120124428.6154f23f@wp.localdomain
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Mon, 20 Nov 2017 00:23:23 +0100
Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> wrote:

> On 11/15/2017 02:13 PM, Robert Haas wrote:
> > On Wed, Nov 15, 2017 at 4:09 AM, Ildus Kurbangaliev
> > <i(dot)kurbangaliev(at)postgrespro(dot)ru> wrote:
> >> So in the next version of the patch I can just unlink the options
> >> from compression methods and dropping compression method will not
> >> affect already compressed tuples. They still could be
> >> decompressed.
> >
> > I guess I don't understand how that can work. I mean, if somebody
> > removes a compression method - i.e. uninstalls the library - and you
> > don't have a way to make sure there are no tuples that can only be
> > uncompressed by that library - then you've broken the database.
> > Ideally, there should be a way to add a new compression method via
> > an extension ... and then get rid of it and all dependencies
> > thereupon.
> I share your confusion. Once you do DROP COMPRESSION METHOD, there
> must be no remaining data compressed with it. But that's what the
> patch is doing already - it enforces this using dependencies, as
> usual.
> Ildus, can you explain what you meant? How could the data still be
> decompressed after DROP COMPRESSION METHOD, and possibly after
> removing the .so library?

The removal of the .so library will broke all compressed tuples. I
don't see a way to avoid it. I meant that DROP COMPRESSION METHOD could
remove the record from 'pg_compression' table, but actually the
compressed tuple needs only a record from 'pg_compression_opt' where
its options are located. And there is dependency between an extension
and the options so you can't just remove the extension without CASCADE,
postgres will complain.

Still it's a problem if the user used for example `SELECT
<compressed_column> INTO * FROM *` because postgres will copy compressed
tuples, and there will not be any dependencies between destination and
the options.

Also thank you for review. I will look into it today.

Ildus Kurbangaliev
Postgres Professional:
Russian Postgres Company

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message atorikoshi 2017-11-20 10:35:11 Failed to delete old ReorderBuffer spilled files
Previous Message Anthony Bykov 2017-11-20 09:35:55 Re: Jsonb transform for pl/python