Re: enhancing pg_basebackup speeds up to ~23Gbps (small fixes + io_uring/Direct I/O)

On 07/10/2026 12:55, Jakub Wartak wrote:
> On Tue, Oct 6, 2026 at 5:42 PM Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>
>> I think that comment is wrong. A short read is possible for other
>> reasons than a truncated file, e.g. if the read is interrupted by a
>> signal. In the past, we've made that same assumption that you never get
>> a short read on a BLCKSZ-sized read in other places too, but it was
>> always questionable. Thomas Munro fixed it for smgrread() in commit
>> 4908c58720.
>>
>> I think we should now fix that in read_file_data_into_buffer(), too.
>> Make it retry, so that it never returns fewer bytes than requested
>> except for EOF. With a larger buffer, a short read becomes more likely.
>
> OK, I see, thanks, so I've tried this in the attached 0004.

Ok, looks good as far as it goes. But looking at basebackup_read_file(),
I think there's a race condition with partial_read_ok == false. This
could happen:

0. read_file_data_into_buffer() observes a checksum failure thanks to a
torn write, and retries reading a single BLCKSZ block.
1. basebackup_read_file() is called with partial_read_ok == false,
nbytes=BLCKSZ.
2. It reads the first 4096 bytes of the block. (e.g. because of
SIMULATE_SHORT_READ, or EINTR)
3. The file is truncated
4. basebackup_read_file() makes another pg_pread() call to read the rest
of the block. It returns 0, because the file was truncated
5. basebackup_read_file() throws a partial read error: "could not read
file \"%s\": read %zu of %zu".

If the truncation happened earlier, before the first read, or later,
after the whole block has been read, you don't get the error.

- Heikki