stat() vs ERROR_DELETE_PENDING, round N + 1

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Cc: Juan José Santamaría Flecha <juanjo(dot)santamaria(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Michael Paquier <michael(at)paquier(dot)xyz>
Subject: stat() vs ERROR_DELETE_PENDING, round N + 1
Date: 2021-09-01 22:10:41
Message-ID: CA+hUKGJz_pZTF9mckn6XgSv69+jGwdgLkxZ6b3NWGLBCVjqUZA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Back 2017, Michael and Magus apparently fixed a bug report[1] about
failing basebackups on Windows due to its concurrent file access
semantics:

commit 9951741bbeb3ec37ca50e9ce3df1808c931ff6a6
Author: Magnus Hagander <magnus(at)hagander(dot)net>
Date: Wed Jan 4 10:48:30 2017 +0100

Attempt to handle pending-delete files on Windows

I think this has been re-broken by:

commit bed90759fcbcd72d4d06969eebab81e47326f9a2
Author: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Date: Fri Oct 9 16:20:12 2020 -0400

Fix our Windows stat() emulation to handle file sizes > 4GB.

There's code in there that appears to understand the
ERROR_PENDING_DELETE stuff, but it seems to be too late, as this bit
will fail with ERROR_ACCESS_DENIED first:

/* fast not-exists check */
if (GetFileAttributes(name) == INVALID_FILE_ATTRIBUTES)
{
_dosmaperr(GetLastError());
return -1;
}

... and if you comment that out, then the CreateFile() call will fail
and we'll return before we get to the code that purports to grok
pending deletes. I don't really understand that code, but I can
report that it's not reached.

This came up because in our work on AIO, we have extra io worker
processes that might have file handles open even in a single session
scenario like 010_pg_basebackup.pl, so we make these types of problems
more likely to hit (hence also my CF entry to fix a related problem in
DROP TABLESPACE). But that's just chance: I assume basebackup could
fail for anyone in 14 for the same reason due to any other backend
that hasn't processed a sinval to close the file yet.

Perhaps we need some combination of the old way (that apparently knew
how to detect pending deletes) and the new way (that knows about large
files)?

[1] https://www.postgresql.org/message-id/flat/20160712083220.1426.58667%40wrigleys.postgresql.org

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2021-09-01 22:29:48 Re: libpq compression
Previous Message Daniel Gustafsson 2021-09-01 22:03:58 Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]