Re: Non-reproducible AIO failure

Hi,

On 2025-08-26 15:21:34 +1200, Thomas Munro wrote:
> On Tue, Aug 26, 2025 at 12:45 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > On 2025-08-25 10:43:21 +1200, Thomas Munro wrote:
> > > On Mon, Aug 25, 2025 at 6:11 AM Konstantin Knizhnik <knizhnik(at)garret(dot)ru> wrote:
> > > > In theory even replacing bitfield with in should not
> > > > avoid race condition, because they are still shared the same cache line.
> > >
> > > I'm no expert in this stuff, but that's not my understanding of how it
> > > works. Plain stores to normal memory go into the store buffer and are
> > > eventually flushed to the memory hierarchy, but all modifications that reach
> > > the cache hierarchy have a consistent view of memory created by the cache
> > > coherency protocol (in ARM's case MOESI[1]): only one core can change a
> > > cache line at a time while it has exclusive access (with some optimisations,
> > > owner mode, snooping, etc but AFAIK that doesn't change the basic
> > > consistency).
> >
> > From what I understand that's not quite right - the whole point of the store
> > buffer is to avoid the latency hit of having to wait for cacheline
> > ownership. Instead the write is done into the store buffer, notably on a
> > granularity *smaller* than the cacheline (it has to be smaller, because we
> > don't have the contents of the cacheline). The reason that that is somewhat
> > OK from a coherency perspective is that this is done only for pure writes, not
> > read-modify-write operations. As the write overwrites the prior contents of
> > the memory, it is "ok" to do the write without waiting for cacheline ownership
> > ahead of time.
>
> *confused* Where's the contradiction?

Maybe I just misunderstood you. I was reading your message as saying that MESI
governs all writes to cachelines, but that's not really true for writes going
through the store buffer... You can write to a cacheline that currently is
exclusively owned by another core.

Greetings,

Andres Freund