pgsql: Make group commit more effective.

From: Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi>
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Make group commit more effective.
Date: 2012-01-30 14:55:07
Message-ID: E1RrsdP-0005K7-PI@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

Make group commit more effective.

When a backend needs to flush the WAL, and someone else is already flushing
the WAL, wait until it releases the WALInsertLock and check if we still need
to do the flush or if the other backend already did the work for us, before
acquiring WALInsertLock. This helps group commit, because when the WAL flush
finishes, all the backends that were waiting for it can be woken up in one
go, and the can all concurrently observe that they're done, rather than
waking them up one by one in a cascading fashion.

This is based on a new LWLock function, LWLockWaitUntilFree(), which has
peculiar semantics. If the lock is immediately free, it grabs the lock and
returns true. If it's not free, it waits until it is released, but then
returns false without grabbing the lock. This is used in XLogFlush(), so
that when the lock is acquired, the backend flushes the WAL, but if it's
not, the backend first checks the current flush location before retrying.

Original patch and benchmarking by Peter Geoghegan and Simon Riggs, although
this patch as committed ended up being very different from that.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/9b38d46d9f5517dab67dda1dd0459683fc9cda9f

Modified Files
--------------
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlog.c | 36 ++++++--
src/backend/storage/lmgr/lwlock.c | 166 +++++++++++++++++++++++++++++++-
src/backend/storage/lmgr/proc.c | 4 +-
src/backend/utils/probes.d | 2 +
src/include/storage/lwlock.h | 6 +-
src/include/storage/proc.h | 2 +-
7 files changed, 200 insertions(+), 18 deletions(-)

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Tom Lane 2012-01-30 15:45:48 Re: pgsql: Resolve timing issue with logging locks for Hot Standby.
Previous Message Simon Riggs 2012-01-30 14:37:47 pgsql: Minor bug fix and cleanup from self-review of sync rep queues pa

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2012-01-30 14:57:27 Re: Group commit, revised
Previous Message Abhijit Menon-Sen 2012-01-30 14:54:13 Re: JSON for PG 9.2