Re: Refactoring the checkpointer's fsync request queue

From: Andres Freund <andres(at)anarazel(dot)de>
To: Kevin Grittner <kgrittn(at)gmail(dot)com>
Cc: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Shawn Debnath <sdn(at)amazon(dot)com>
Subject: Re: Refactoring the checkpointer's fsync request queue
Date: 2019-01-22 21:01:52
Message-ID: 20190122210152.gfeak6basvbvfkon@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On 2019-01-22 14:53:11 -0600, Kevin Grittner wrote:
> On Tue, Jan 22, 2019 at 2:38 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> > close() doesn't trigger an fsync() in general
>
> What sort of a performance hit was observed when testing the addition
> of fsync or fdatasync before any PostgreSQL close() of a writable
> file, or have we not yet checked that?

I briefly played with it, and it was so atrocious (as in, less than
something like 0.2x the throughput) that I didn't continue far down that
path. Two ways I IIRC (and it's really just memory) I tried were:

a) Short lived connections that do a bunch of writes to files each. That
turns each disconnect into an fsync of most files.
b) Workload with > max_files_per_process files (IIRC I just used a bunch
of larger tables with a few indexes each) in a read/write workload
that's a bit larger than shared buffers. That lead to most file
closes being integrity writes, with obvious downsides.

Greetings,

Andres Freund

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2019-01-22 21:11:23 Re: [PATCH] Pass COPT and PROFILE to CXXFLAGS as well
Previous Message Kevin Grittner 2019-01-22 20:53:11 Re: Refactoring the checkpointer's fsync request queue