Re: [HACKERS] Custom compression methods

On Mon, Jun 22, 2020 at 4:53 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > Or maybe we add 1 or 2 "privileged" built-in compressors that get
> > dedicated bit-patterns in the upper 2 bits of the size field, with the
> > last bit pattern being reserved for future algorithms. (e.g. 0x00 =
> > pglz, 0x01 = lz4, 0x10 = zstd, 0x11 = something else - see within for
> > details).
>
> Agreed. I favor an approach roughly like I'd implemented below
> https://postgr.es/m/20130605150144.GD28067%40alap2.anarazel.de
> I.e. leave the vartag etc as-is, but utilize the fact that pglz
> compressed datums starts with a 4 byte length header, and that due to
> the 1GB limit, the first two bits currently have to be 0. That allows to
> indicate 2 compression methods without any space overhead, and
> additional compression methods are supported by using an additional byte
> (or some variable length encoded larger amount) if both bits are 1.

I think there's essentially no difference between these two ideas,
unless the two bits we're talking about stealing are not the same in
the two cases. Am I missing something?

> One additional note: Adding additional vartag_external values does incur
> some noticable cost, distributed across lots of places.

OK.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company