Re: Spread checkpoint sync

Robert Haas wrote:
> On Tue, Nov 30, 2010 at 3:29 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
>
>> One of the ideas Simon and I had been considering at one point was adding
>> some better de-duplication logic to the fsync absorb code, which I'm
>> reminded by the pattern here might be helpful independently of other
>> improvements.
>>
>
> Hopefully I'm not stepping on any toes here, but I thought this was an
> awfully good idea and had a chance to take a look at how hard it would
> be today while en route from point A to point B. The answer turned
> out to be "not very", so PFA a patch that seems to work. I tested it
> by attaching gdb to the background writer while running pgbench, and
> it eliminate the backend fsyncs without even breaking a sweat.
>

No toe damage, this is great, I hadn't gotten to coding for this angle
yet at all. Suffering from an overload of ideas and (mostly wasted)
test data, so thanks for exploring this concept and proving it works.

I'm not sure what to do with the rest of the work I've been doing in
this area here, so I'm tempted to just combine this new bit from you
with the older patch I submitted, streamline, and see if that makes
sense. Expected to be there already, then "how about spending 5 minutes
first checking out that autovacuum lock patch again" turned out to be a
wild underestimate.

Part of the problem is that it's become obvious to me the last month
that right now is a terrible time to be doing Linux benchmarks that
impact filesystem sync behavior. The recent kernel changes that are
showing in the next rev of the enterprise distributions--like RHEL6 and
Debian Squeeze both working to get a stable 2.6.32--have made testing a
nightmare. I don't want to dump a lot of time into optimizing for
<2.6.32 if this problem changes its form in newer kernels, but the
distributions built around newer kernels are just not fully baked enough
yet to tell. For example, the pre-release Squeeze numbers we're seeing
are awful so far, but it's not really done yet either. I expect 3-6
months from today, that all will have settled down enough that I can
make some sense of it. Lately my work with the latest distributions has
just been burning time installing stuff that doesn't work quite right yet.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books