Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

From: Andres Freund <andres(at)anarazel(dot)de>
To: pgsql-hackers(at)postgresql(dot)org
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>
Subject: Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up
Date: 2010-05-21 05:11:34
Message-ID: 201005210711.35247.andres@anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Friday 21 May 2010 05:40:03 Robert Haas wrote:
> On Thu, May 20, 2010 at 4:27 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> > I looked a bit around for faster implementations of CRC32 and found one
> > in zlib. After adapting it (pg uses slightly different computation (non-
> > inverted)) I found that it increases the speed of the CRC32 calculation
> > itself 3 fold.
>
> But zlib is not under the PostgreSQL license.
Yes. But:
1. the zlib license shouldn't be a problem in itself - pg_dump also already
links to zlib
2. I planned to ask Mark Adler whether he would support relicising those bits.
I have read some other discussions where he was supportive of doing such a
thing
3. Given that idea was posted publically on the usenet it is not hard to
produce an independent implementation.

So I do not see any big problems there... Or am I missing something?

Greetings,

Andres

/* zlib.h -- interface of the 'zlib' general purpose compression library
version 1.2.2, October 3rd, 2004

Copyright (C) 1995-2004 Jean-loup Gailly and Mark Adler

This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.

Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:

1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.

Jean-loup Gailly jloup(at)gzip(dot)org
Mark Adler madler(at)alumni(dot)caltech(dot)edu

*/

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Kevin Flanagan 2010-05-21 08:19:11 Re: ERROR: GIN indexes do not support whole-index scans
Previous Message Mohammad Heykal Abdillah 2010-05-21 04:22:48 Re: Why SELECT keyword on parser is written as SELECTME ?