Re: Checkpoint cost, looks like it is WAL/CRC

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: josh(at)agliodbs(dot)com
Cc: "Zeugswetter Andreas DAZ SD" <ZeugswetterA(at)spardat(dot)at>, "Simon Riggs" <simon(at)2ndquadrant(dot)com>, "Bruce Momjian" <pgman(at)candle(dot)pha(dot)pa(dot)us>, "Bruno Wolff III" <bruno(at)wolff(dot)to>, "Greg Stark" <gsstark(at)mit(dot)edu>, "Russell Smith" <mr-russ(at)pws(dot)com(dot)au>, "Postgres Hackers" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Checkpoint cost, looks like it is WAL/CRC
Date: 2005-07-07 13:38:08
Message-ID: 7278.1120743488@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I wrote:
> We still don't know enough about the situation to know what a solution
> might look like. Is the slowdown Josh is seeing due to the extra CPU
> cost of the CRCs, or the extra I/O cost, or excessive locking of the
> WAL-related data structures while we do this stuff, or ???. Need more
> data.

Josh, is OSDL up enough that you can try another comparison run?
If so, please undo the previous patch (which disabled page dumping
entirely) and instead try removing this block of code, starting
at about xlog.c line 620 in CVS tip:

/*
* Now add the backup block headers and data into the CRC
*/
for (i = 0; i < XLR_MAX_BKP_BLOCKS; i++)
{
if (dtbuf_bkp[i])
{
BkpBlock *bkpb = &(dtbuf_xlg[i]);
char *page;

COMP_CRC32(rdata_crc,
(char *) bkpb,
sizeof(BkpBlock));
page = (char *) BufferGetBlock(dtbuf[i]);
if (bkpb->hole_length == 0)
{
COMP_CRC32(rdata_crc,
page,
BLCKSZ);
}
else
{
/* must skip the hole */
COMP_CRC32(rdata_crc,
page,
bkpb->hole_offset);
COMP_CRC32(rdata_crc,
page + (bkpb->hole_offset + bkpb->hole_length),
BLCKSZ - (bkpb->hole_offset + bkpb->hole_length));
}
}
}

This will remove just the CRC calculation work associated with backed-up
pages. Note that any attempt to recover from the WAL will fail, but I
assume you don't need that for the purposes of the test run.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2005-07-07 13:51:11 Re: A couple of patches for PostgreSQL 64bit support
Previous Message Tom Lane 2005-07-07 13:30:57 Re: Checkpoint cost, looks like it is WAL/CRC