Re: refactoring basebackup.c (zstd workers)

On Mon, Mar 14, 2022 at 1:11 PM Justin Pryzby <pryzby(at)telsasoft(dot)com> wrote:
> Internally, I was thinking they'd all be handled as first-class options, with
> separate struct fields and separate replication protocol options. If an option
> isn't known, it'd be rejected on the client side, rather than causing an error
> on the server.

There's some appeal to that, but one downside is that it means that
the client can't be used to fetch data that is compressed in a way
that the server knows about and the client doesn't. I don't think
that's great. Why should, for example, pg_basebackup need to be
compiled with zstd support in order to request zstd compression on the
server side? If the server knows about the brand new
justin-magic-sauce compression algorithm, maybe the client should just
be able to request it and, when given various .jms files by the
server, shrug its shoulders and accept them for what they are. That
doesn't work if -Fp is involved, or similar, but it should work fine
for simple cases if we set things up right.

> Maybe there'd be an option parser for this in common/ (I think that might
> require having new data structure there too, maybe one for each compression
> method, or maybe a union{} to handles them all). Most of the ~100 lines to
> support wal_compression='zstd:N' are to parse out the N.

Yes, it's actually a very simple feature now that we've got the rest
of the infrastructure set up correctly for it.

--
Robert Haas
EDB: http://www.enterprisedb.com