Re: O(n^2) system calls in RemoveOldXlogFiles()

Hi,

Thanks for pushing! Sorry to not get around to a review before...

On 2021-01-15 11:42:50 +0900, Michael Paquier wrote:
> On Fri, Jan 15, 2021 at 03:25:24PM +1300, Thomas Munro wrote:
> > Thanks Michael! Another notch for the unnecessary system call
> > hitlist: https://wiki.postgresql.org/wiki/Syscall_Reduction
>
> A quick question. How much does it matter in terms of
> micro-performance for this code path depending on max/min_wal_size?

I can't see it being a problem here. Disision-by-non-constant is
expensive, but syscalls are more expensive. And journalled filesystem
operations like renames are *much* more expensive.

> Andres has mentioned its aio work, without telling any numbers.

I only found it while working on the AIO stuff, it's not really
dependant on it. I saw significant performance drops in the last part of
a checkpoint in both aio / master, but additional debugging output I had
in the aio branch made it obvious that it's not actuall the buffer sync
where the time is spent.

> "No backpatch is done per the lack of field complaints."

Because there's no real way to attribute the slowdown to WAL file
recycling in production workloads on master, I don't think we could
really expect field complaints. Everyone will just attribute the
slowdown to BufferSync() or file sync.

I think the way we currently emit WAL timings is quite quite
unhelpful. The fact that we attribute CheckpointWriteDelay() to the
write time makes it nearly useless until you're at the point the
checkpoint can't be completed in time. Similarly, the "sync" time"
covers many things that aren't syncing...

Greetings,

Andres Freund