Checkpointing question

From: "Strange, John W" <john(dot)w(dot)strange(at)jpmchase(dot)com>
To: "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: Checkpointing question
Date: 2011-02-15 11:45:28
Message-ID: EF37296944B47C40ADDCCB7BFD6289FE048ADB9704@EMASC201VS01.exchad.jpmchase.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

During heavy writes times we get the checkpoint too often error, what's the real knock down effect of checkpointing too often? The documents don't really say what is wrong with checkpointing too often, does it cause block, io contention, etc, etc? From my understanding it's just IO contention, but I wanted to make sure.

14.4.6. Increase checkpoint_segments
Temporarily increasing the checkpoint_segments<http://www.postgresql.org/docs/8.4/static/runtime-config-wal.html#GUC-CHECKPOINT-SEGMENTS> configuration variable can also make large data loads faster. This is because loading a large amount of data into PostgreSQL will cause checkpoints to occur more often than the normal checkpoint frequency (specified by the checkpoint_timeout configuration variable). Whenever a checkpoint occurs, all dirty pages must be flushed to disk. By increasing checkpoint_segments temporarily during bulk data loads, the number of checkpoints that are required can be reduced.

- John

[ 2011-02-15 09:34:30.549 GMT ] :4d404b6e.47ef LOG: checkpoint starting: xlog
[ 2011-02-15 09:34:43.656 GMT ] :4d404b6e.47ef LOG: checkpoint complete: wrote 36135 buffers (0.4%); 0 transaction log file(s) added, 0 removed, 12 recycled; write=13.101 s, sync=0.000 s, total=13.107 s
[ 2011-02-15 09:34:57.090 GMT ] :4d404b6e.47ef LOG: checkpoints are occurring too frequently (27 seconds apart)
[ 2011-02-15 09:34:57.090 GMT ] :4d404b6e.47ef HINT: Consider increasing the configuration parameter "checkpoint_segments".
[ 2011-02-15 09:34:57.090 GMT ] :4d404b6e.47ef LOG: checkpoint starting: xlog
[ 2011-02-15 09:35:11.492 GMT ] :4d404b6e.47ef LOG: checkpoint complete: wrote 54634 buffers (0.7%); 0 transaction log file(s) added, 0 removed, 30 recycled; write=14.290 s, sync=0.000 s, total=14.401 s
[ 2011-02-15 09:35:25.496 GMT ] :4d404b6e.47ef LOG: checkpoints are occurring too frequently (28 seconds apart)
[ 2011-02-15 09:35:25.496 GMT ] :4d404b6e.47ef HINT: Consider increasing the configuration parameter "checkpoint_segments".
[ 2011-02-15 09:35:25.496 GMT ] :4d404b6e.47ef LOG: checkpoint starting: xlog
[ 2011-02-15 09:35:39.688 GMT ] :4d404b6e.47ef LOG: checkpoint complete: wrote 39352 buffers (0.5%); 0 transaction log file(s) added, 0 removed, 30 recycled; write=14.185 s, sync=0.000 s, total=14.192 s
[ 2011-02-15 09:35:53.417 GMT ] :4d404b6e.47ef LOG: checkpoints are occurring too frequently (28 seconds apart)
[ 2011-02-15 09:35:53.417 GMT ] :4d404b6e.47ef HINT: Consider increasing the configuration parameter "checkpoint_segments".
[ 2011-02-15 09:35:53.417 GMT ] :4d404b6e.47ef LOG: checkpoint starting: xlog
[ 2011-02-15 09:36:09.059 GMT ] :4d404b6e.47ef LOG: checkpoint complete: wrote 48803 buffers (0.6%); 0 transaction log file(s) added, 0 removed, 30 recycled; write=15.408 s, sync=0.000 s, total=15.641 s

This communication is for informational purposes only. It is not
intended as an offer or solicitation for the purchase or sale of
any financial instrument or as an official confirmation of any
transaction. All market prices, data and other information are not
warranted as to completeness or accuracy and are subject to change
without notice. Any comments or statements made herein do not
necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
and affiliates.

This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.

Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to European legal entities.

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Kim A. Brandt 2011-02-15 14:23:40 LIMIT on partitioned-table!?
Previous Message Steve Atkins 2011-02-14 16:43:24 Re: How to boost performance of queries containing pattern matching characters