Re: [PERFORM] WAL Optimisation - configuration and usage

From: "Simon Riggs" <simon(at)2ndquadrant(dot)com>
To: "'Tom Lane'" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "'Rob Fielding'" <rob(at)dsvr(dot)net>, <pgsql-performance(at)postgresql(dot)org>, <pgsql-hackers-pitr(at)postgresql(dot)org>
Subject: Re: [PERFORM] WAL Optimisation - configuration and usage
Date: 2004-03-02 22:16:12
Message-ID: 004801c400a3$f9696640$5baa87d9@LaptopDellXP
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-hackers-pitr pgsql-performance

>Tom Lane
> "Simon Riggs" <simon(at)2ndquadrant(dot)com> writes:
> > - why checkpoint interval of 300 secs causes them to happen every 10
> > mins in quieter periods; is that an occaisional update occurring?
>
> There is code in there to suppress a checkpoint if no WAL-loggable
> activity has happened since the last checkpoint. Not sure if that's
> relevant to the issue or not though.

Thanks Tom, at least that clears up why the checkpoints are off.

I must admit, I'm taken aback though:

I'd prefer it if it DIDN'T do that. If the system is quiet, the odd
checkpoint doesn't matter that much - however, taking a long time to
return the xlog files to the *desired* state of having many
pre-allocated log files is not a great thing.

What do you think about continuing to checkpoint normally until the
number of xlog files has returned to 2*checkpoint_segments+1, then
allowing a slow down of checkpoints when quiet? It would be easy enough
to set a variable true while rearranging files to the limit, then set it
false when the limit has been hit and then using that to activate the
slow-down code (not that I know where that is mind...). However, that
would require some backend to postmaster ipc, which may be a problem.

Or perhaps the real problem is only recycling one file at a time - if
we're running this as the checkpoint process it wouldn't be a problem to
recycle more than one at the same time would it?

The reason for my interest is: when archiving logs for PITR, there may
be occasional long pauses while waiting for tape mounts (typically 30
minutes from notification to change). These pauses could (and therefore
will eventually for some people) cause severe log file build up, and I'm
interested in making sure this build up doesn't take too long to clear.
Forgetting the archival API stuff for a second, this is roughly the same
situation as Rob is experiencing (or at least causing him to pause and
think).

Best Regards, Simon Riggs

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Simon Riggs 2004-03-02 22:53:09 Re: Out of space situation and WAL log pre-allocation (was Tablespaces)
Previous Message Joe Conway 2004-03-02 21:56:16 Re: IN and ANY

Browse pgsql-hackers-pitr by date

  From Date Subject
Next Message Neil Conway 2004-03-03 00:01:11 Re: WAL Optimisation - configuration and usage
Previous Message Raymond Siebert 2004-03-02 14:20:08 Re: Archival API

Browse pgsql-performance by date

  From Date Subject
Next Message scott.marlowe 2004-03-02 22:36:52 Re: Scaling further up
Previous Message Anjan Dave 2004-03-02 22:03:32 Re: Scaling further up