Re: What exactly is our CRC algorithm?

On 2014-11-21 13:01:50 +0200, Heikki Linnakangas wrote:
> On 11/21/2014 12:11 PM, Abhijit Menon-Sen wrote:
> >At 2014-11-20 13:47:00 +0530, ams(at)2ndQuadrant(dot)com wrote:
> >>
> >>>Suggestions for how to address (b) are welcome.
> >
> >With help from Andres, I set up a workload where XLogInsert* was at the
> >top of my profiles: server with fsync and synchronous_commit off, and
> >pgbench running a multiple-row insert into a single-text-column table
> >with -M prepared -c 25 -t 250000 -f script.
>
> How wide is the row, in terms of bytes? You should see bigger improvement
> with wider rows, as you get longer contiguous chunks of data to CRC that
> way. With very narrow rows, you might not see much difference because the
> chunks are so small.

The primary goal, as I understood it, was to test very short records to
test whether there's a regression due to slice-by-8. There doesn't seem
to be any.

It's, IIRC, trivial to reproduce significant performance benefits in
workloads with lots of FPWs or large records (like COPY), but those
weren't what you and Tom were concerned about...

> If that's the problem, it might be beneficial to memcpy() all the data to a
> temporary buffer, and calculate the CRC over the whole, instead of CRC'ing
> each XLogRecData chunk separately. XLogRecordAssemble() uses a scratch area,
> hdr_scratch, for building all the headers. You could check how much
> rmgr-specific data there is, and if there isn't much, just append the data
> to that scratch area too.

I think that might very well make sense - but it's imo something better
done separately from the smarter CRC algorithm.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services