Re: Fsync request queue

On 2018-05-01 13:43:14 -0400, Robert Haas wrote:
> On Tue, May 1, 2018 at 1:41 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> > I unfortunately don't have access to the relevant reports anymore, so
> > it's only by memory. What I do remember is that a few I saw
> > pg_stat_bgwriter.buffers_backend_fsync values that we a pretty sizable
> > fraction of the buffers written by backends. I don't think I ever
> > figured out how problematic that was from a peformance perspective, and
> > how large a fraction of the overall number of fsyncs those were.
> >
> > One was a workload with citus (lots of tables per node), and one was
> > inheritance based partitioning. There were a few others too, where I
> > don't recall anything about the workload.
>
> Hmm. Partitioning probably does make it easier to overrun the queue,
> but even so it seems hard -- the queue has one entry per shared
> buffer, which is a lot.

Yea, I really don't remember the details unfortunately. I guess if you
have a large number of tables and then a large number of corresponding
relations (indexes, sequences) and there's some temporal locality of
which tables are accessed, it's not insane to think you could exceed
NBuffers relations. As I said, I'm not sure whether this caused actual
performance issues, just that I saw the higher value (there were enough
architectual issues to fix...).

Greetings,

Andres Freund