Re: Direct I/O

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:

> On 2023-04-12 We 10:23, Dagfinn Ilmari Mannsåker wrote:
>> Andrew Dunstan<andrew(at)dunslane(dot)net> writes:
>>
>>> On 2023-04-12 We 01:48, Thomas Munro wrote:
>>>> On Wed, Apr 12, 2023 at 3:04 PM Thomas Munro<thomas(dot)munro(at)gmail(dot)com> wrote:
>>>>> On Wed, Apr 12, 2023 at 2:56 PM Christoph Berg<myon(at)debian(dot)org> wrote:
>>>>>> I'm hitting a panic in t_004_io_direct. The build is running on
>>>>>> overlayfs on tmpfs/ext4 (upper/lower) which is probably a weird
>>>>>> combination but has worked well for building everything over the last
>>>>>> decade. On Debian unstable:
>>>>>>
>>>>>> PANIC: could not open file "pg_wal/000000010000000000000001": Invalid argument
>>>>> ... I have a new idea: perhaps it is possible to try
>>>>> to open a file with O_DIRECT from perl, and if it fails like that,
>>>>> skip the test. Looking into that now.
>>>> I think I have that working OK. Any Perl hackers want to comment on
>>>> my use of IO::File (copied from examples on the internet that showed
>>>> how to use O_DIRECT)? I am not much of a perl hacker but according to
>>>> my package manager, IO/File.pm came with perl itself. And the Fcntl
>>>> eval trick that I copied from File::stat, and the perl-critic
>>>> suppression that requires?
>>>
>>> I think you can probably replace a lot of the magic here by simply saying
>>>
>>>
>>> if (Fcntl->can("O_DIRECT")) ...
>> Fcntl->can() is true for all constants that Fcntl knows about, whether
>> or not they are defined for your OS. `defined &O_DIRECT` is the simplest
>> check, see my other reply to Thomas.
>>
>>
>
> My understanding was that Fcntl only exported constants known to the
> OS. That's certainly what its docco suggests, e.g.:
>
>     By default your system's F_* and O_* constants (eg, F_DUPFD and
> O_CREAT)
>     and the FD_CLOEXEC constant are exported into your namespace.

It's a bit more magical than that (this is Perl after all). They are
all exported (which implicitly creates stubs visible to `->can()`,
similarly to forward declarations like `sub O_FOO;`), but only the
defined ones (`#ifdef O_FOO` is true) are defined (`defined &O_FOO` is
true). The rest fall through to an AUTOLOAD¹ function that throws an
exception for undefined ones.

Here's an example (Fcntl knows O_RAW, but Linux does not define it):

$ perl -E '
use strict; use Fcntl;
say "can", main->can("O_RAW") ? "" : "not";
say defined &O_RAW ? "" : "not ", "defined";
say O_RAW;'
can
not defined
Your vendor has not defined Fcntl macro O_RAW, used at -e line 4

While O_DIRECT is defined:

$ perl -E '
use strict; use Fcntl;
say "can", main->can("O_DIRECT") ? "" : "not";
say defined &O_DIRECT ? "" : "not ", "defined";
say O_DIRECT;'
can
defined
16384

And O_FOO is unknown to Fcntl (the parens on `O_FOO()q are to make it
not a bareword, which would be a compile error under `use strict;` when
the sub doesn't exist at all):

$ perl -E '
use strict; use Fcntl;
say "can", main->can("O_FOO") ? "" : "not";
say defined &O_FOO ? "" : "not ", "defined";
say O_FOO();'
cannot
not defined
Undefined subroutine &main::O_FOO called at -e line 4.

> cheers
>
>
> andrew

- ilmari

[1] https://perldoc.perl.org/perlsub#Autoloading