Re: Acclerating INSERT/UPDATE using UPS

From: "Simon Riggs" <simon(at)2ndquadrant(dot)com>
To: "Bruce Momjian" <bruce(at)momjian(dot)us>
Cc: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>, "Christopher Browne" <cbbrowne(at)acm(dot)org>, <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Acclerating INSERT/UPDATE using UPS
Date: 2007-02-26 21:37:42
Message-ID: 1172525863.3760.393.camel@silverbirch.site
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, 2007-02-26 at 09:44 -0500, Bruce Momjian wrote:
> Joshua D. Drake wrote:
> > Christopher Browne wrote:
> > > A long time ago, in a galaxy far, far away, kawasima(at)cs(dot)tsukuba(dot)ac(dot)jp (Hideyuki Kawashima) wrote:
> > >> I appreciate your great suggestion!
> > >> It is great honor for me if Sigres will be merged to PostgreSQL.
> > >> Since the changes of Sigres from PostgreSQL-8.2.1 are not many,
> > >> and moreover, all of changes are surrounded with #ifdef SIGRES --- #endif,
> > >> incorporating Sigres into PostgreSQL would be easy.
> > >
> > > You should consider submitting a patch for this against CVS HEAD.
> > >
> > > And actually, I'd think it a better idea to define a GUC variable and
> > > use that to control whether Sigres is active or not.
> > >
> > > At the more sophisticated end of the spectrum, you might set things up
> > > so that it could be activated/deactivated at runtime by a superuser.
> > >
> > > At the less sophisticated end, it might need to be configured in
> > > postgresql.conf...
> >
> > Whatever happen with this?
>
> I would like to see more analysis about why Sigres is faster than an
> in-memory file system. I think the idea was that locking was reduced
> but I am unclear on why locking is different in the two cases.

Reading through the design, I see the following

- bgwriter performs XLogWrite, not each backend
- WAL fsync is only performed when WAL file fills
- no checkpoints are performed until shutdown

This is approximately the same performance as fsync=off, with a big
boost because we never do checkpoints either (i.e. 10% as stated by the
OP). There is a slight gain because of reduced lock contention, but I
wouldn't expect that to be major.

This comes back to the idea of deferred fsync we've spoken about a few
times, to provide "MySQL mode" performance and reduced robustness
guarantees. The idea of having a special backend perform the XLogWrites
rather than the individual backends is already a TODO item, even if this
implementation is somewhat more extreme in its approach to fsync delay.
The actual potential data loss is similar to an archive recovery though,
so I do like that touch - i.e. we might lose everything in the current
WAL file.

IMHO, it shouldn't be the bgwriter that fsyncs WAL because it cannot
clean shared_buffers and fsync the WAL concurrently with any
effectiveness. Perhaps such infrequent fsyncs mean thats not a problem.

Not checkpointing at all is not a good plan, since this will lead to an
enormous build up of WAL files and a very long recovery time if the
system does fail.

Overall, these choices effect the whole system whereas what I think we
need is something that can be applied to just certain transactions or
tables.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Warren Turkal 2007-02-26 21:48:49 Re: [Monotone-devel] Re: SCMS question
Previous Message Joshua D. Drake 2007-02-26 21:28:08 Re: Proposal for Implenting read-only queries during wal replay (SoC 2007)