Skip site navigation (1) Skip section navigation (2)

Re: Load distributed checkpoint

From: ITAGAKI Takahiro <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>
To: "Takayuki Tsunakawa" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>
Cc: <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Load distributed checkpoint
Date: 2006-12-21 09:46:36
Message-ID: 20061221181709.60BE.ITAGAKI.TAKAHIRO@oss.ntt.co.jp (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-patches
"Takayuki Tsunakawa" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com> wrote:

> > If you use Linux, it has very unpleased behavior in fsync(); It locks all
> > metadata of the file being fsync-ed. We have to wait for the completion of
> > fsync when we do read(), write(), and even lseek().
> 
> Oh, really, what an evil fsync is!  Yes, I sometimes saw a backend
> waiting for lseek() to complete when it committed.  But why does the
> backend which is syncing WAL/pg_control have to wait for syncing the
> data file?  They are, not to mention, different files, and WAL and
> data files are stored on separate disks.

Backends call lseek() in planning, so they have to wait fsync() to
the table that they will access. Even if all of data in the file is in
the cache, lseek() conflict with fsync(). You can see a lot of backends
are waiting in planning phase in checkpoints, not executing phase.


> > it's not difficult at all to insert sleeps between fsync()s.

> Can we say "our approach helps our
> environments, but it won't help you.  The kernel VM settings may help
> you.  Good luck!"?

I didn't say such a thing at all.
There are several opinions in the discussion:
  1. High bgwriter setting is enough!
  2. Change your OS :-)
  3. Use O_SYNC or O_DIRECT, but very poor performance.
  4. We may settle for single fsync(), but not many fsync()s in a short time.
I just suggested 4.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center



In response to

Responses

pgsql-hackers by date

Next:From: Martijn van OosterhoutDate: 2006-12-21 10:26:41
Subject: Re: Load distributed checkpoint
Previous:From: Bernd HelmleDate: 2006-12-21 08:32:43
Subject: Re: Release 8.2.0 done, 8.3 development starts

pgsql-patches by date

Next:From: Mark Morgan LloydDate: 2006-12-21 09:47:09
Subject: Re: AGAIN - problem with BCC55 and libpq 8.2
Previous:From: Takayuki TsunakawaDate: 2006-12-21 08:14:13
Subject: Re: Load distributed checkpoint

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group