Re: Time to up bgwriter_lru_maxpages?

From: Andres Freund <andres(at)anarazel(dot)de>
To: Jim Nasby <Jim(dot)Nasby(at)BlueTreble(dot)com>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Time to up bgwriter_lru_maxpages?
Date: 2017-02-02 19:50:35
Message-ID: 20170202195035.wbobgydm4go32bt3@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

On 2017-02-02 11:41:44 -0800, Jim Nasby wrote:
> On 2/1/17 4:28 PM, Andres Freund wrote:
> > On 2016-11-28 11:40:53 -0800, Jim Nasby wrote:
> > > With current limits, the most bgwriter can do (with 8k pages) is 1000 pages
> > > * 100 times/sec = 780MB/s. It's not hard to exceed that with modern
> > > hardware. Should we increase the limit on bgwriter_lru_maxpages?
> >
> > FWIW, I think working on replacing bgwriter (e.g. by working on the
> > patch I send with a POC replacement) wholesale is a better approach than
> > spending time increasing limits.
>
> Do you have a link to that? I'm not seeing anything in the archives.

Not at hand, but I can just give you the patches. These are very likely
to conflict, but it shouldn't be too hard to resolve...

What it basically does is move as much of the clock-sweep to bgwriter,
which keeps a certain number of clean pages available. There's a
lock-free ringbuffer that backends can just pick pages off.

The approach with the ringbuffer has the advantage that with relatively
small changes we can scale to having multiple bgwriters (needs some
changes in the ringbuf implementation, but not that many).

Andres

Attachment Content-Type Size
0001-Basic-lockless-single-producer-multiple-consumer-rin.patch text/x-patch 6.3 KB
0002-WIP-bgwriter-rewrite.patch text/x-patch 23.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2017-02-02 19:52:39 Re: Time to up bgwriter_lru_maxpages?
Previous Message Robert Haas 2017-02-02 19:47:53 Re: Time to up bgwriter_lru_maxpages?