Re: Large files for relations

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Large files for relations
Date: 2023-05-03 05:36:42
Message-ID: CA+hUKGLP81Hvjbtzfr9c0V50dF+2pB2pOwvQ4MsWdU8-pa_H7Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, May 3, 2023 at 5:21 PM Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:
> rsync --link-dest

I wonder if rsync will grow a mode that can use copy_file_range() to
share blocks with a reference file (= previous backup). Something
like --copy-range-dest. That'd work for large-file relations
(assuming a file system that has block sharing, like XFS and ZFS).
You wouldn't get the "mtime is enough, I don't even need to read the
bytes" optimisation, which I assume makes all database hackers feel a
bit queasy anyway, but you'd get the space savings via the usual
rolling checksum or a cheaper version that only looks for strong
checksum matches at the same offset, or whatever other tricks rsync
might have up its sleeve.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message samay sharma 2023-05-03 06:39:52 Re: Overhauling "Routine Vacuuming" docs, particularly its handling of freezing
Previous Message Thomas Munro 2023-05-03 05:21:14 Re: Large files for relations