Re: refactoring basebackup.c

From: Jeevan Ladhe <jeevanladhe(dot)os(at)gmail(dot)com>
To: tushar <tushar(dot)ahuja(at)enterprisedb(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: "Shinoda, Noriyoshi (PN Japan FSIP)" <noriyoshi(dot)shinoda(at)hpe(dot)com>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, Dipesh Pandit <dipesh(dot)pandit(at)gmail(dot)com>, Abhijit Menon-Sen <ams(at)toroid(dot)org>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Jeevan Ladhe <jeevan(dot)ladhe(at)enterprisedb(dot)com>, Mark Dilger <mark(dot)dilger(at)enterprisedb(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: refactoring basebackup.c
Date: 2022-02-15 18:03:50
Message-ID: CANm22CivaxViV4hok-jwfZQQxa6_6noZF803D4sFCT-jQDTFaA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thanks Tushar for the testing.

I further worked on ZSTD and now have implemented client side
compression as well. Attached are the patches for both server-side and
client-side compression.

The patch 0001 is a server-side patch, and has not changed since the
last patch version - v10, but, just bumping the version number.

Patch 0002 is the client-side compression patch.

Regards,
Jeevan Ladhe

On Tue, 15 Feb 2022 at 22:24, tushar <tushar(dot)ahuja(at)enterprisedb(dot)com> wrote:

> On 2/15/22 6:48 PM, Jeevan Ladhe wrote:
> > Please find the attached updated version of patch for ZSTD server side
> Thanks, Jeevan, I again tested with the attached patch, and as mentioned
> the crash is fixed now.
>
> also, I tested with different labels with gzip V/s zstd against data
> directory size which is 29GB and found these results
>
> ====
> ./pg_basebackup -t server:/tmp/<directory>
> --compress=server-zstd:<label> -Xnone -n -N --no-estimate-size -v
>
> --compress=server-zstd:1 = compress directory size is 1.3GB
> --compress=server-zstd:4 = compress directory size is 1.3GB
> --compress=server-zstd:7 = compress directory size is 1.2GB
> --compress=server-zstd:12 = compress directory size is 1.2GB
> ====
>
> ===
> ./pg_basebackup -t server:/tmp/<directooy>
> --compress=server-gzip:<label> -Xnone -n -N --no-estimate-size -v
>
> --compress=server-gzip:1 = compress directory size is 1.8GB
> --compress=server-gzip:4 = compress directory size is 1.6GB
> --compress=server-gzip:9 = compress directory size is 1.6GB
> ===
>
> --
> regards,tushar
> EnterpriseDB https://www.enterprisedb.com/
> The Enterprise PostgreSQL Company
>
>

Attachment Content-Type Size
v11-0001-Add-a-ZSTD-compression-method-for-server-side-compre.patch application/octet-stream 36.8 KB
v11-0002-ZSTD-CLIENT-compression-WIP-working.patch application/octet-stream 12.2 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2022-02-15 18:10:34 Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work
Previous Message Justin Pryzby 2022-02-15 17:59:44 Re: refactoring basebackup.c (zstd)