Re: pg_upgrade --copy-file-range fails with EINVAL on Linux 4.19

On 26.08.26 17:39, Peter Eisentraut wrote:
> This approach is not correct. You don't know whether the kernel you are
> running against is the one you built against. Most likely it is not.

Sorry for the extra round.
While implementing it I did notice that a build-time header check
cannot cover binaries that later run under an older kernel, but I
stated that as a design fact instead of raising it as an open question,
which is what I should have done.

> I think a possible correct approach would be to somehow do this check
> in the pg_upgrade --check phase.

Attached is v2.
It replaces the header-version check with a runtime probe.
Instead of comparing uname(2) against 5.3, I made
check_copy_file_range() run the same loop that copyFileByRange() runs:

do
{
nbytes = copy_file_range(src_fd, NULL, dest_fd, NULL, SSIZE_MAX, 0);
if (nbytes < 0)
pg_fatal("could not copy file range between old and new data directories: %m");
}
while (nbytes > 0);

The probe file (PG_VERSION) is a few bytes, so this exercises exactly
what the transfer path does for every non-empty relation file:
one call that copies the whole file, then one more call that must
return 0 rather than fail.
The mode still cannot copy relation files on 4.19; the failure now
happens in the check phase, so the user can pick --copy / --link /
--clone before transfer starts.
I picked a behavioral probe over a version check because behavior does
not always follow the version number:
a vendor could backport the 5.3 handling into an older kernel, where a
version check would wrongly disable the mode while the probe correctly
keeps it working; and a future regression would be caught by the probe
but not by the version check.
It also reuses the existing probe and adds no platform-specific code.

On Linux 4.19.0-21-amd64 (Debian 10, ext4) a user who asks for
--copy-file-range is now stopped in the check phase with
"could not copy file range between old and new data directories".
Relation files are not copied.
HAVE_COPY_FILE_RANGE is defined, so this is not
"copy_file_range not supported on this platform".
t/006_transfer_modes.pl already treats that check-phase message as an
expected failure, so the test passes without changes.

On Linux 6.6.87.2 (Ubuntu 24.04 WSL2, ext4) --copy-file-range completes
as before.

Both builds are PostgreSQL 20devel at de5659440db plus this file.c
change.
Same command, test file not changed:

make -C src/bin/pg_upgrade check PROVE_TESTS='t/006_transfer_modes.pl'

# 4.19
t/006_transfer_modes.pl .. ok
Files=1, Tests=40
Result: PASS

# 6.6
t/006_transfer_modes.pl .. ok
Files=1, Tests=50
Result: PASS

The extra tests on 6.6 are the post-upgrade data checks that 4.19 does
not reach.

> But maybe nothing needs to be done. You get a clean error return if
> what you are doing is not supported by copy_file_range(), so that
> seems ok?

That is a fair view of the error itself: the mode is opt-in and the
failure is clean.
I do not think we should leave it as is.
The check phase currently passes, then transfer fails on the extra
copy_file_range() call with a different message.
Passing the check phase does not mean the mode will work.
On such kernels the test suite goes red for the same reason.
Making the probe run the same loop as transfer is the smaller fix.

pg_combinebackup has the same do-while with NULL + SSIZE_MAX in
copy_file_by_range() (src/bin/pg_combinebackup/copy_file.c) and no
check phase of its own, so on 4.19 it would still fail mid-run.
I left that alone in this patch: there is no --check to align, and
the failure is already loud.
The other call sites (reconstruct.c, and copydir.c in the backend) do
not use that do-while with NULL + SSIZE_MAX, so they are not affected.

Thanks,
Johnny

Peter Eisentraut <peter(at)eisentraut(dot)org> 于2026年8月26日周三 23:39写道：
>
> On 23.08.26 13:54, 达劳里亚斯 wrote:
> > Here is a patch for option (4): do not define HAVE_COPY_FILE_RANGE
> > when building against Linux kernel headers older than 5.3.
>
> This approach is not correct. You don't know whether the kernel you are
> running against is the one you built against. Most likely it is not.
>
> I think a possible correct approach would be to somehow do this check in
> the pg_upgrade --check phase.
>
> But maybe nothing needs to be done. You get a clean error return if
> what you are doing is not supported by copy_file_range(), so that seems ok?
>