pg_upgrade --copy-file-range

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: pg_upgrade --copy-file-range
Date: 2023-06-02 19:30:44
Message-ID: CA+hUKGKe7Hb0-UNih8VD5UNZy5-ojxFb3Pr3xSBBL8qj2M2=dQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

I was just in a pg_upgrade unconference session at PGCon where the
lack of $SUBJECT came up. This system call gives the kernel the
option to use fast block cloning on XFS, ZFS (as of very recently),
etc, and works on Linux and FreeBSD. It's probably much the same as
--clone mode on COW file systems, except that is Linux-only. On
overwrite file systems (ie not copy-on-write, like ext4), it may also
be able to push copies down to storage hardware/network file systems.

There was something like this in the nearby large files patch set, but
in that version it just magically did it when available in --copy
mode. Now I think the user should have to have to opt in with
--copy-file-range, and simply to error out if it fails. It may not
work in some cases -- for example, the man page says that older Linux
systems can fail with EXDEV when you try to copy across file systems,
while newer systems will do something less efficient but still
sensible internally; also I saw a claim that some older versions had
weird bugs. Better to just expose the raw functionality and let users
say when they want it and read the error if it fail, I think.

Attachment Content-Type Size
0001-Add-copy-file-range-option-to-pg_upgrade.patch text/x-patch 9.8 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ankit Pandey 2023-06-02 19:42:32 [Question] pgwal increasing over max_wal_size
Previous Message James Coleman 2023-06-02 17:51:22 Re: RFC: Logging plan of the running query