Re: Non-reproducible AIO failure

Hi,

On 2025-06-12 15:12:00 +0300, Konstantin Knizhnik wrote:
> Reproduced it once again with with write-protected io handle.
> But once again - no access violation, just assert failure.
>
> Previously "op" field was overwritten somewhere between `pgaio_io_reclaim`
> and `AsyncReadBuffers`:
>
> !!!pgaio_io_reclaim [20376]| ioh: 0x1019bc000, ioh->op: 0, ioh->generation:
> 19346
> !!!AsyncReadBuffers [20376] (1)| blocknum: 21, ioh: 0x1019bc000, ioh->op: 1,
> ioh->state: 1, ioh->result: 0, ioh->num_callbacks: 0, ioh->generation: 19346
>
> Now it is overwritten after print in AsyncReadBuffers:
>
> !!!pgaio_io_reclaim [88932]| ioh: 0x105a5c000, ioh->op: 0, ioh->generation:
> 42848
> !!!pgaio_io_acquire_nb[88932]| ioh: 0x105a5c000, ioh->op: 0,
> ioh->generation: 42848
> !!!AsyncReadBuffers [88932] (1)| blocknum: 10, ioh: 0x105a5c000, ioh->op: 0,
> ioh->state: 1, ioh->result: 0, ioh->num_callbacks: 0, ioh->generation: 42848
> !!!pgaio_io_before_start| ioh: 0x105a5c000, ioh->op: 1, ioh->state: 1,
> ioh->result: 0, ioh->num_callbacks: 2, ioh->generation: 42848
>
> In this run I prohibit writes to io handle in `pgaio_io_acquire_nb` and
> reenable them in `AsyncReadBuffer`.

I'm reasonably certain I found the issue, I think it's a missing memory
barrier on the read side. The CPU is reordering the read (or just using a
cached value) of ->distilled_result to be before the load of ->state.

But it'll take a bit to verify that that's the issue...

Greetings,

Andres Freund