Re: Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

From: Andres Freund <andres(at)anarazel(dot)de>
To: Greg Stark <gsstark(at)mit(dot)edu>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Bruce Momjian <bruce(at)momjian(dot)us>, pgsql-hackers(at)postgresql(dot)org, singh(dot)gurjeet(at)gmail(dot)com
Subject: Re: Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up
Date: 2010-05-30 19:43:43
Message-ID: 201005302143.43734.andres@anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sunday 30 May 2010 18:29:31 Greg Stark wrote:
> On Sun, May 30, 2010 at 4:54 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > I read through that thread and couldn't find much discussion of
> > alternative CRC implementations --- we spent all our time on arguing
> > about whether we needed 64-bit CRC or not.
>
> Alright, how about this thread?
>
> http://thread.gmane.org/gmane.comp.db.postgresql.devel.general/71741
>
> This actually sounds like precisely the same algorithm. Perhaps this
> implementation is much better but your tests on the old one showed a
> big difference between smaller and larger data sequences.
I haven't yet had a chance to read the intel paper (I am in the train and
latency is 30s+ and the original link is dead), but I got the sf.net
implementation.

Seeing it I think I might know the reason why it wasn't as much faster as
promised - it introduces ordering constraints by avoiding shifts by using
term2. Not sure though.

Anybody got the implementation by Gurjeet? I couldn't find it online (within
the constraints of the connection).

Greetings,

Andres

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2010-05-30 20:48:37 Re: Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up
Previous Message Jan Urbański 2010-05-30 18:02:05 Re: tsvector pg_stats seems quite a bit off.