| From: | Greg Sabino Mullane <htamfids(at)gmail(dot)com> |
|---|---|
| To: | Dimitrios Apostolou <jimis(at)gmx(dot)net> |
| Cc: | pgsql-general(at)lists(dot)postgresql(dot)org |
| Subject: | Re: In-order pg_dump (or in-order COPY TO) |
| Date: | 2025-08-27 14:06:40 |
| Message-ID: | CAKAnmmLY1EVyV-V0Rmb98F63TDHbx4LQFJ5Jv7qK8ourbCoEKQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general |
On Tue, Aug 26, 2025 at 3:44 PM Dimitrios Apostolou <jimis(at)gmx(dot)net> wrote:
> I am storing dumps of a database (pg_dump custom format) in a
> de-duplicating backup server. Each dump is many terabytes in size, so
> deduplication is very important. And de-duplication itself is based on
> rolling checksums which is pretty flexible, it can compensate for blocks
> moving by some offset.
>
I suggest looking into pgBackRest, and it's block incremental feature,
which sounds similar to what you are doing. But it also does it with
parallel processes, and can do things like grab backup files from your
replicas, plus a lot of other features.
Cheers,
Greg
--
Crunchy Data - https://www.crunchydata.com
Enterprise Postgres Software Products & Tech Support
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Lane | 2025-08-27 14:16:33 | Re: In-order pg_dump (or in-order COPY TO) |
| Previous Message | Dimitrios Apostolou | 2025-08-27 12:52:39 | Re: In-order pg_dump (or in-order COPY TO) |