Skip site navigation (1) Skip section navigation (2)

Re: [HACKERS] Full page writes improvement, code update

From: Koichi Suzuki <suzuki(dot)koichi(at)oss(dot)ntt(dot)co(dot)jp>
To: Zeugswetter Andreas ADI SD <ZeugswetterA(at)spardat(dot)at>
Cc: Hannu Krosing <hannu(at)skype(dot)net>, "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Simon Riggs <simon(at)2ndquadrant(dot)com>, josh(at)agliodbs(dot)com, pgsql-hackers(at)postgresql(dot)org, pgsql-patches(at)postgresql(dot)org
Subject: Re: [HACKERS] Full page writes improvement, code update
Date: 2007-04-13 00:09:50
Message-ID: 461ECA4E.9010101@oss.ntt.co.jp (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-patches
Hi,

Sorry, inline reply.

Zeugswetter Andreas ADI SD wrote:

> 
> Yup, this is a good summary.
> 
> You say you need to remove the optimization that avoids 
> the logging of a new tuple because the full page image exists.
> I think we must already have the info in WAL which tuple inside the full
> page image
> is new (the one for which we avoided the WAL entry for).
> 
> How about this:
> Leave current WAL as it is and only add the not removeable flag to full
> pages.
> pg_compresslog then replaces the full page image with a record for the
> one tuple that is changed.
> I tend to think it is not worth the increased complexity only to save
> bytes in the uncompressed WAL though.

It is essentially what my patch proposes.  My patch includes flag to 
full page writes which "can be" removed.

> Another point about pg_decompresslog:
> 
> Why do you need a pg_decompresslog ? Imho pg_compresslog should already
> do the replacing of the
> full_page with the dummy entry. Then pg_decompresslog could be a simple
> gunzip, or whatever compression was used,
> but no logic.

Just removing full page writes does not work.   If we shift the rest of 
the WAL, then LSN becomes inconsistent in compressed archive logs which 
pg_compresslog produces.   For recovery, we have to restore LSN as the 
original WAL.   Pg_decompresslog restores removed full page writes as a 
dumm records so that recovery redo functions won't be confused.

Regards;

> 
> Andreas
> 


-- 
-------------
Koichi Suzuki

In response to

Responses

pgsql-hackers by date

Next:From: Tom LaneDate: 2007-04-13 00:20:06
Subject: Re: Strangely Variable Query Performance
Previous:From: SteveDate: 2007-04-13 00:05:48
Subject: Re: Strangely Variable Query Performance

pgsql-patches by date

Next:From: Tom LaneDate: 2007-04-13 00:20:06
Subject: Re: Strangely Variable Query Performance
Previous:From: SteveDate: 2007-04-13 00:05:48
Subject: Re: Strangely Variable Query Performance

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group