Re: AdvanceXLInsertBuffers() vs wal_sync_method=open_datasync

On 10/11/2023 05:54, Andres Freund wrote:
> In this case I had used wal_sync_method=open_datasync - it's often faster and
> if we want to scale WAL writes more we'll have to use it more widely (you
> can't have multiple fdatasyncs in progress and reason about which one affects
> what, but you can have multiple DSYNC writes in progress at the same time).

Not sure I understand that. If you issue an fdatasync, it will sync all
writes that were complete before the fdatasync started. Right? If you
have multiple fdatasyncs in progress, that's true for each fdatasync. Or
is there a bottleneck in the kernel with multiple in-progress fdatasyncs
or something?

> After a bit of confused staring and debugging I figured out that the problem
> is that the RequestXLogSwitch() within the code for starting a basebackup was
> triggering writing back the WAL in individual 8kB writes via
> GetXLogBuffer()->AdvanceXLInsertBuffer(). With open_datasync each of these
> writes is durable - on this drive each take about 1ms.

I see. So the assumption in AdvanceXLInsertBuffer() is that XLogWrite()
is relatively fast. But with open_datasync, it's not.

> To fix this, I suspect we need to make
> GetXLogBuffer()->AdvanceXLInsertBuffer() flush more aggressively. In this
> specific case, we even know for sure that we are going to fill a lot more
> buffers, so no heuristic would be needed. In other cases however we need some
> heuristic to know how much to write out.

+1. Maybe use the same logic as in XLogFlush().

I wonder if the 'flexible' argument to XLogWrite() is too inflexible. It
would be nice to pass a hard minimum XLogRecPtr that it must write up
to, but still allow it to write more than that if it's convenient.

--
Heikki Linnakangas
Neon (https://neon.tech)