From: | Bruce Momjian <bruce(at)momjian(dot)us> |
---|---|
To: | ITAGAKI Takahiro <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp> |
Cc: | pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: Load distributed checkpoint |
Date: | 2007-02-02 22:54:58 |
Message-ID: | 200702022254.l12MswU04166@momjian.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers pgsql-patches |
Thread added to TODO list:
* Reduce checkpoint performance degredation by forcing data to disk
more evenly
http://archives.postgresql.org/pgsql-hackers/2006-12/msg00337.php
http://archives.postgresql.org/pgsql-hackers/2007-01/msg00079.php
---------------------------------------------------------------------------
ITAGAKI Takahiro wrote:
> This is a proposal for load distributed checkpoint.
> (It is presented on postgresql anniversary summit in last summer.)
>
>
> We offen encounters performance gap during checkpoint. The reason is write
> bursts. Storage devices are too overworked in checkpoint, so they can not
> supply usual transaction processing.
>
> Checkpoint consists of the following four steps, and the major performance
> problem is 2nd step. All dirty buffers are written without interval in it.
>
> 1. Query information (REDO pointer, next XID etc.)
> 2. Write dirty pages in buffer pool
> 3. Flush all modified files
> 4. Update control file
>
> I suggested to write pages with sleeping in 2nd step, using normal activity
> of the background writer. It is something like cost-based vacuum delay.
> Background writer has two pointers, 'ALL' and 'LRU', indicating where to
> write out in buffer pool. We can wait for the ALL clock-hand going around
> to guarantee all pages to be written.
>
> Here is pseudo-code for the proposed method. The internal loop is just the
> same as bgwriter's activity.
>
> PrepareCheckPoint(); -- do step 1
> Reset num_of_scanned_pages by ALL activity;
> do {
> BgBufferSync(); -- do a part of step 2
> sleep(bgwriter_delay);
> } while (num_of_scanned_pages < shared_buffers);
> CreateCheckPoint(); -- do step 3 and 4
>
>
> We may accelerate background writer to reduce works at checkpoint instead of
> the method, but it introduces another performance problem; Extra pressure
> is always put on the storage devices to keep the number of dirty pages low.
>
>
> I'm working about adjusting the progress of checkpoint to checkpoint timeout
> and wal segments limitation automatically to avoid overlap of two checkpoints.
> I'll post a patch sometime soon.
>
> Comments and suggestions welcome.
>
> Regards,
> ---
> ITAGAKI Takahiro
> NTT Open Source Software Center
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
From | Date | Subject | |
---|---|---|---|
Next Message | Bruce Momjian | 2007-02-02 22:58:06 | Re: [pgsql-patches] [GENERAL] Corrupt database? 8.1/FreeBSD6.0 |
Previous Message | Tom Lane | 2007-02-02 21:50:10 | Re: Referential Integrity and SHARE locks |
From | Date | Subject | |
---|---|---|---|
Next Message | Bruce Momjian | 2007-02-02 22:58:06 | Re: [pgsql-patches] [GENERAL] Corrupt database? 8.1/FreeBSD6.0 |
Previous Message | Andrew Dunstan | 2007-02-02 21:26:56 | Re: pre-vcbuild win32 install docs incorrect |