Re: CLOG extension

From: Simon Riggs <simon(at)2ndQuadrant(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: CLOG extension
Date: 2012-05-03 17:27:44
Message-ID: CA+U5nMJM0e706SbaGR3zv2fCszkhO7HNz7Z_u1K+d-RJSOmTDA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, May 3, 2012 at 5:59 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> [ CLOG extension is horrid for concurrency ]
>
> Yeah.  When that code was designed, a page's worth of transactions
> seemed like a lot so we didn't worry too much about performance glitches
> when we crossed a page boundary.  It's time to do something about it
> though.
>
> The idea of extending CLOG in advance, so that the work doesn't have to
> be done with quite so many locks held, sounds like a plan to me.  The
> one thing I'd worry about is that extension has to interact with
> freezing of very old XIDs and subsequent removal of old clog pages;
> make sure that pages will get removed before they could possibly
> get created again.
>
>> First, do we really need to WAL-log CLOG extension at all?  Perhaps
>> recovery should simply extend CLOG when it hits a commit or abort
>> record that references a page that doesn't exist yet.
>
> Maybe, but see above.  I'd be particularly worried about this in a hot
> standby situation, as you would then end up with HS queries seeing XIDs
> (in tuples) for which there was no clog page yet.  I'm inclined to think
> it's better to continue to WAL-log it, but try to arrange to do that
> without holding the other locks that are now involved.

Why not switch to 1 WAL record per file, rather than 1 per page. (32
pages, IIRC).

We can then have the whole new file written as zeroes by a background
process, which needn't do that while holding the XidGenLock.

My earlier patch to do background flushing from bgwriter can be
extended to do that.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2012-05-03 17:28:43 Re: Advisory locks seem rather broken
Previous Message Robert Haas 2012-05-03 17:22:24 Re: Advisory locks seem rather broken