Re: where should I stick that backup?

On Mon, Apr 20, 2020 at 8:18 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Sat, Apr 18, 2020 at 8:35 PM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> >
> > On Fri, Apr 17, 2020 at 7:44 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > > This suggest that pipes do have a considerably higher overhead on
> > > windows, but that it's not all that terrible if one takes care to use
> > > large buffers in each pipe element.
> > >
> > > It's notable though that even the simplest use of a pipe does add a
> > > considerable overhead compared to using the files directly.
> >
> > Thanks for these results. I think that this shows that it's probably
> > not a great idea to force everything to go through pipes in every
> > case, but on the other hand, there's no reason to be a particularly
> > scared of the performance implications of letting some things go
> > through pipes. For instance, if we decide that LZ4 compression is
> > going to be a good choice for most users, we might want to do that
> > in-process rather than via pipes.
> >
>
> How will the user know how to use this compressed backup? I mean to
> say if we use some compression algorithm to compress the data then the
> user should know how to decompress and use the backup. IIUC, if
> currently, the user uses tar format to backup, it can simply untar it
> and start the server but will that be possible if we provide some
> in-built compression methods like LZ4?
>

One idea could be that we can write something like BACKUP COMPRESSION:
<LZ4 or whatever compression we have used> in backup_label file and
then probably recovery can take care of decompressing it.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com