Skip site navigation (1) Skip section navigation (2)

Re: COMMIT NOWAIT Performance Option

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: "Jonah H(dot) Harris" <jonah(dot)harris(at)gmail(dot)com>
Cc: Gregory Stark <stark(at)enterprisedb(dot)com>, Josh Berkus <josh(at)agliodbs(dot)com>, Jeff Davis <pgsql(at)j-davis(dot)com>, pgsql-hackers(at)postgresql(dot)org, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Subject: Re: COMMIT NOWAIT Performance Option
Date: 2007-02-28 17:16:10
Message-ID: 200702281716.l1SHGAP25360@momjian.us (view raw or flat)
Thread:
Lists: pgsql-hackers
I think we need to think about when these CRCs would be read and
written.  It would be written when it hits the disk, hopefully by the
background writer, and I think after a server crash, all pages would
have to be read and checked.  The good news is that both of these are
non-critical paths.

---------------------------------------------------------------------------

Jonah H. Harris wrote:
> On 2/28/07, Gregory Stark <stark(at)enterprisedb(dot)com> wrote:
> > But we've already seen that CRC checks can be expensive. Not everyone will
> > want to take the cpu hit. Storing a byte counter in every block is cheap.
> 
> CRC checking a page is most certainly the simplest.  And, I disagree
> that it would be worse than either a sequence counter or the full page
> write.  Block checksumming is done at read/write time... which is
> something that needs to be improved anyway.  With a properly tuned
> bgwriter, the write itself should barely be noticeable.  How fast is a
> CRC of 8K?  Last time I checked it was something on the scale of ~95
> usec for CRC32 and ~33 usec for sb8.
> 
> > And the idea came from what someone said MSSQL does, so "like everyone else"
> > -- which isn't a very compelling argument to begin with -- doesn't argue
> > against it.
> 
> Rather than basing designs on poor second-hand information, maybe you
> and the person who mentioned this idea should get up-to-date and read
> the SQL Server storage engine architecture.
> 
> As of SQL Server 2005, blocks *are* checksummed with CRC32.  And, just
> for the record, previous versions of SQL server performed a bit
> flipping technique for every 512 bytes in the page header; it did
> *not* waste a byte for every 512 bytes written.
> 
> > I think the way you would work is to have the smgr note the sequential value
> > it found when it read in a page and then when it writes it out increment that
> > value by one. Conveniently the pages would be 16 bytes shorter than an 8kb
> > page so you have 16 bytes available with every buffer to note information like
> > the last sequential tag the buffer used.
> 
> This proposed design is overcomplicated and a waste of space.  I mean,
> we reduce storage overhead using phantom command id and variable
> varlena, but let's just fill it up again with unnecessary junk bytes.
> 
> > That seems pretty unlikely. CRC checks are expensive cpu-wise, we're already
> > suffering a copy due to our use of read/write the difference between
> > read/write of 8192 bytes and readv/writev of 511b*16+1*6 is going to be
> > non-zero but very small. Thousands of times quicker than the CRC.
> 
> Prove it.
> 
> -- 
> Jonah H. Harris, Software Architect | phone: 732.331.1324
> EnterpriseDB Corporation            | fax: 732.331.1301
> 33 Wood Ave S, 3rd Floor            | jharris(at)enterprisedb(dot)com
> Iselin, New Jersey 08830            | http://www.enterprisedb.com/
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
>        choose an index scan if your joining column's datatypes do not
>        match

-- 
  Bruce Momjian  <bruce(at)momjian(dot)us>          http://momjian.us
  EnterpriseDB                               http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

In response to

Responses

pgsql-hackers by date

Next:From: Gregory StarkDate: 2007-02-28 17:16:39
Subject: Re: COMMIT NOWAIT Performance Option
Previous:From: Sergey E. KoposovDate: 2007-02-28 17:14:13
Subject: Re: SOC & user quotas

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group