Skip site navigation (1) Skip section navigation (2)

Re: Full page writes improvement, code update

From: "Simon Riggs" <simon(at)2ndquadrant(dot)com>
To: "Koichi Suzuki" <suzuki(dot)koichi(at)oss(dot)ntt(dot)co(dot)jp>
Cc: <pgsql-hackers(at)postgresql(dot)org>,<pgsql-patches(at)postgresql(dot)org>
Subject: Re: Full page writes improvement, code update
Date: 2007-03-27 21:07:49
Message-ID: 1175029670.4386.222.camel@silverbirch.site (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-patches
On Tue, 2007-03-27 at 11:52 +0900, Koichi Suzuki wrote:

> Here's an update of a code to improve full page writes as proposed in
> 
> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01491.php
> and
> http://archives.postgresql.org/pgsql-patches/2007-01/msg00607.php
> 
> Update includes some modification for error handling in archiver and
> restoration command.
> 
> In the previous threads, I posted several evaluation and shown that we
> can keep all the full page writes needed for full XLOG crash recovery,
> while compressing archive log size considerably better than gzip, with
> less CPU consumption.  I've found no further objection for this proposal
> but still would like to hear comments/opinions/advices.

Koichi-san,

Looks interesting. I like the small amount of code to do this.

A few thoughts:

- Not sure why we need "full_page_compress", why not just mark them
always? That harms noone. (Did someone else ask for that? If so, keep
it)

- OTOH I'd like to see an explicit parameter set during recovery since
you're asking the main recovery path to act differently in case a single
bit is set/unset. If you are using that form of recovery, we should say
so explicitly, to keep everybody else safe.

- I'd rather mark just the nonremovable blocks. But no real difference

- We definitely don't want an normal elog in a XLogInsert critical
section, especially at DEBUG1 level

- diff -c format is easier and the standard

- pg_logarchive and pg_logrestore should be called by a name that
reflects what they actually do. Possibly pg_compresslog and
pg_decompresslog etc.. I've not reviewed those programs, but:

- Not sure why we have to compress away page headers. Touch as little as
you can has always been my thinking with recovery code.

- I'm very uncomfortable with touching the LSN. Maybe I misunderstand?

- Have you thought about how pg_standby would integrate with this
option? Can you please?

- I'll do some docs for this after Freeze, if you'd like. I have some
other changes to make there, so I can do this at the same time.

-- 
  Simon Riggs             
  EnterpriseDB   http://www.enterprisedb.com



In response to

Responses

pgsql-hackers by date

Next:From: Andrew DunstanDate: 2007-03-27 21:11:01
Subject: Re: Concurrent connections in psql
Previous:From: Andrew DunstanDate: 2007-03-27 21:04:39
Subject: Re: Arrays of Complex Types

pgsql-patches by date

Next:From: Gregory StarkDate: 2007-03-27 21:40:13
Subject: Re: [PATCH] add CLUSTER table ORDER BY index
Previous:From: Andrew DunstanDate: 2007-03-27 21:04:39
Subject: Re: Arrays of Complex Types

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group