From: | Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi> |
---|---|
To: | pgsql-committers(at)postgresql(dot)org |
Subject: | pgsql: Improve scalability of WAL insertions. |
Date: | 2013-07-08 09:16:32 |
Message-ID: | E1Uw7Ye-0002ao-0i@gemulon.postgresql.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-committers pgsql-hackers |
Improve scalability of WAL insertions.
This patch replaces WALInsertLock with a number of WAL insertion slots,
allowing multiple backends to insert WAL records to the WAL buffers
concurrently. This is particularly useful for parallel loading large amounts
of data on a system with many CPUs.
This has one user-visible change: switching to a new WAL segment with
pg_switch_xlog() now fills the remaining unused portion of the segment with
zeros. This potentially adds some overhead, but it has been a very common
practice by DBA's to clear the "tail" of the segment with an external
pg_clearxlogtail utility anyway, to make the WAL files compress better.
With this patch, it's no longer necessary to do that.
This patch adds a new GUC, xloginsert_slots, to tune the number of WAL
insertion slots. Performance testing suggests that the default, 8, works
pretty well for all kinds of worklods, but I left the GUC in place to allow
others with different hardware to test that easily. We might want to remove
that before release.
Reviewed by Andres Freund.
Branch
------
master
Details
-------
http://git.postgresql.org/pg/commitdiff/9a20a9b21baa819df1760b36f3c36f25d11fc27b
Modified Files
--------------
src/backend/access/transam/xlog.c | 2111 ++++++++++++++++++++++++++++---------
src/backend/storage/lmgr/spin.c | 2 +
src/backend/utils/misc/guc.c | 11 +
src/include/access/xlog.h | 1 +
src/include/access/xlogdefs.h | 10 -
src/include/storage/lwlock.h | 2 +-
6 files changed, 1641 insertions(+), 496 deletions(-)
From | Date | Subject | |
---|---|---|---|
Next Message | Heikki Linnakangas | 2013-07-08 10:07:40 | pgsql: Fix pg_test_fsync, broken by xloginsert scaling patch. |
Previous Message | Tom Lane | 2013-07-08 02:37:55 | pgsql: Fix planning of parameterized appendrel paths with expensive joi |
From | Date | Subject | |
---|---|---|---|
Next Message | Heikki Linnakangas | 2013-07-08 09:16:34 | Re: XLogInsert scaling, revisited |
Previous Message | Pavel Stehule | 2013-07-08 09:13:50 | Re: Improving avg performance for numeric |