Re: [HACKERS] Custom compression methods

I'm a minor contributor now to a couple bits of this patch set, but I can
answer a couple of these points.

On Mon, Mar 15, 2021 at 03:58:35PM -0700, Andres Freund wrote:
> Comments about 0003:
> - why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be
> comparable to HIDE_TABLEAM?

That was my idea and implementation.
It's because until 3 weeks ago, the patchset supported a "plugable compression
API" like CREATE ACCESS METHOD, a suggestion from Alvaro to avoid making a new
table and everything involved just for a few rows). Now, the patch is limited
to lz4, and the "pluggable compression APIs" isn't included in the latest
patchsets.

> Comments about 0005:
> - I'm personally not really convinced tracking the compression type in
> pg_attribute the way you do is really worth it (. Especially given
> that it's right now only about new rows anyway. Seems like it'd be
> easier to just treat it as a default for new rows, and dispense with
> all the logic around mismatching compression types etc?

I made the half-serious suggestion to make it a per-relation relopt.
That would allow implementing pg_dump --no-toast-compression, to allow
restoring a dump from a server with LZ4 tables to a server --without-lz4.
Similar to --no-tablespaces.

That would also avoid adding a compression column in \d (which avoids the need
for HIDE_TOAST_COMPRESSION).

> > I'm open to being convinced that we don't need to do either of these
> > things, and that the cost of iterating over all varlenas in the tuple
> > is not so bad as to preclude doing things as you have them here. But,
> > I'm afraid it's going to be too expensive.
>
> I mean, I would just define several of those places away by not caring
> about tuples in a different compressino formation ending up in a
> table...

If I understand you right, this is because it's desirable to allow 1) migrating
existing data from pglz to lz4; 2) also allow moving away from lz4, if need be.

--
Justin