Re: Checkpointer write combining

I continued reviewing v16, focusing on checkpointer and bgwriter.

In 0004:

> + CheckpointWriteDelay(flags, (double) num_processed / num_to_scan,
> + processed);

This is now called once per batch instead of once per processed buffer,
but each call can still sleep for at most 100 ms. This can make smaller
spread checkpoints finish much earlier.

On Linux, with shared_buffers = 128MB, fsync = off,
checkpoint_timeout = 30s and checkpoint_completion_target = 0.9,
I updated a 20000-row table after a fast checkpoint, then issued
CHECKPOINT (MODE spread). Each run wrote 1146 buffers:

pg_stat_io.writes write phase
master 1146 26.869s
v16, io_combine_limit=8kB 1146 26.166s
v16, default 128kB limit 75 7.121s
v16, per-buffer delay calls 75 26.259s

For the last row I restored per-buffer delay calls after releasing the
batch's locks. Could we preserve that pacing when combining writes?

In 0006:

> + for (; num_to_scan > 0; num_to_scan--, next_to_clean++)
> ...
> + if (++num_written >= bgwriter_lru_maxpages)
> {
> + PendingBgWriterStats.maxwritten_clean++;
> + break;
> }

Reaching the limit skips the for-loop update. The last buffer is
counted as reusable but omitted from scan progress. The saved position
still points to it. Advancing and wrapping the scan position after
selecting the buffer, before processing it, should avoid this.
The issue survives in BgwriterWriteBuffers() later in the series.

In 0018:

> + GatherContiguousDirtyBuffers(bufHdr, IOCONTEXT_NORMAL, &batch);
> + WriteBuffers(&batch);
> + CompleteWriteBuffers(&batch, wb_context);
> + num_written += batch.n;
>
> + if (num_written >= lru_maxpages)

The combined write can already exceed bgwriter_lru_maxpages by the time
we check it. Calling run_bgwriter_cleaner(1) in the existing test
writes all six contiguous buffers. Could GatherContiguousDirtyBuffers()
also take the remaining per-round budget, so the documented limit
still holds?

Thank you!

Best regards, Andrey Borodin.