Skip site navigation (1) Skip section navigation (2)

Re: Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

From: Andres Freund <andres(at)anarazel(dot)de>
To: Greg Stark <gsstark(at)mit(dot)edu>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Bruce Momjian <bruce(at)momjian(dot)us>, pgsql-hackers(at)postgresql(dot)org, singh(dot)gurjeet(at)gmail(dot)com
Subject: Re: Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up
Date: 2010-05-30 19:43:43
Message-ID: 201005302143.43734.andres@anarazel.de (view raw or flat)
Thread:
Lists: pgsql-hackers
On Sunday 30 May 2010 18:29:31 Greg Stark wrote:
> On Sun, May 30, 2010 at 4:54 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > I read through that thread and couldn't find much discussion of
> > alternative CRC implementations --- we spent all our time on arguing
> > about whether we needed 64-bit CRC or not.
> 
> Alright, how about this thread?
> 
> http://thread.gmane.org/gmane.comp.db.postgresql.devel.general/71741
> 
> This actually sounds like precisely the same algorithm. Perhaps this
> implementation is much better but your tests on the old one showed a
> big difference between smaller and larger data sequences.
I haven't yet had a chance to read the intel paper (I am in the train and 
latency is 30s+ and the original link is dead), but I got the sf.net 
implementation.

Seeing it I think I might know the reason why it wasn't as much faster as 
promised - it introduces ordering constraints by avoiding shifts by using 
term2. Not sure though.

Anybody got the implementation by Gurjeet? I couldn't find it online (within 
the constraints of the connection).

Greetings,

Andres

In response to

Responses

pgsql-hackers by date

Next:From: Andres FreundDate: 2010-05-30 20:48:37
Subject: Re: Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up
Previous:From: Jan UrbaƄskiDate: 2010-05-30 18:02:05
Subject: Re: tsvector pg_stats seems quite a bit off.

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group