From: | Michael Paquier <michael(at)paquier(dot)xyz> |
---|---|
To: | Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> |
Cc: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: file cloning in pg_upgrade and CREATE DATABASE |
Date: | 2018-03-19 07:06:36 |
Message-ID: | 20180319070636.GA4633@paquier.xyz |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Tue, Feb 20, 2018 at 10:00:04PM -0500, Peter Eisentraut wrote:
> Some new things have happened since then:
>
> - XFS has (optional) reflink support. This file system is probably more
> widely used than Btrfs.
Btrfs is still in development, there are I think no many people who
would use it in production.
> - Linux and glibc have a proper function to do this now.
>
> - APFS on macOS supports file cloning.
So copyfile() is only part of macos? I am not able to find references
in FreeBSD, NetBSD or OpenBSD, but I may be missing something.
> So altogether this feature will be more widely usable and less ugly to
> implement. Note, however, that you will currently need literally the
> latest glibc release, so it probably won't be accessible right now
> unless you are using Fedora 28 for example. (This is the
> copy_file_range() function that had us recently rename the same function
> in pg_rewind.)
For reference, Debian SID is using glibc 2.27. ArchLinux is still on
2.26.
> Some example measurements:
>
> 6 GB database, pg_upgrade unpatched 30 seconds, patched 3 seconds (XFS
> and APFS)
Interesting. I'll try to test that on an XFS partition and see if I can
see a difference. For now I have just read through the patch.
+#ifdef HAVE_COPYFILE
+ if (copyfile(fromfile, tofile, NULL,
+#ifdef COPYFILE_CLONE
+ COPYFILE_CLONE
+#else
+ COPYFILE_DATA
+#endif
+ ) < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not copy file \"%s\" to \"%s\": %m", fromfile, tofile)));
+#else
copy_file(fromfile, tofile);
+#endif
Any backend-side callers of copy_file() would not benefit from
copyfile() on OSX. Shouldn't all that handling be inside copy_file(),
similarly to what your patch actually does for pg_upgrade? I think that
you should also consider fcopyfile() instead of copyfile() as it works
directly on the file descriptors and share the same error handling as
the others.
--
Michael
From | Date | Subject | |
---|---|---|---|
Next Message | Kyotaro HORIGUCHI | 2018-03-19 07:11:39 | Re: User defined data types in Logical Replication |
Previous Message | Amit Khandekar | 2018-03-19 06:55:06 | Re: Hash join in SELECT target list expression keeps consuming memory |