pgsql: aio: Stop using enum bitfields due to bad code generation

aio: Stop using enum bitfields due to bad code generation

During an investigation into rather odd aio related errors on macos, observed
by Alexander and Konstantin, we started to wonder if bitfield access is
related to the error. At the moment it looks like it is related, we cannot
reproduce the failures when replacing the bitfields. In addition, the problem
can only be reproduced with some compiler [versions] and not everyone has been
able to reproduce the issue.

The observed problem is that, very rarely, PgAioHandle->{state,target} are in
an inconsistent state, after having been checked to be in a valid state not
long before, triggering an assertion failure. Unfortunately, this could be
caused by wrong compiler code generation or somehow of missing memory barriers
- we don't really know. In theory there should not be any concurrent write
access to the handle in the state the bug is triggered, as the handle was idle
and is just being initialized.

Separately from the bug, we observed that at least gcc and clang generate
rather terrible code for the bitfield access. Even if it's not clear if the
observed assertion failure is actually caused by the bitfield somehow, the bad
code generation alone is sufficient reason to stop using bitfields.

Therefore, replace the enum bitfields with uint8s and instead cast in each
switch statement.

Reported-by: Alexander Lakhin <exclusion(at)gmail(dot)com>
Reported-by: Konstantin Knizhnik <knizhnik(at)garret(dot)ru>
Discussion: https://postgr.es/m/1500090.1745443021@sss.pgh.pa.us
Backpatch-through: 18

Branch
------
REL_18_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/ce161b194e84c941c468261fb72f10ed3a99b958

Modified Files
--------------
src/backend/storage/aio/aio.c | 10 +++++-----
src/backend/storage/aio/aio_funcs.c | 2 +-
src/backend/storage/aio/aio_io.c | 8 ++++----
src/backend/storage/aio/method_io_uring.c | 2 +-
src/include/storage/aio_internal.h | 14 ++++++++++----
5 files changed, 21 insertions(+), 15 deletions(-)