Re: Spread checkpoint sync
From:
Greg Smith <greg(at)2ndquadrant(dot)com>
To:
Robert Haas <robertmhaas(at)gmail(dot)com>
Cc:
Jeff Janes <jeff(dot)janes(at)gmail(dot)com>,
Ron Mayer <rm_pg(at)cheapcomplexdevices(dot)com>,
Josh Berkus <josh(at)agliodbs(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject:
Re: Spread checkpoint sync
Date:
2011-01-17 03:13:59
Message-ID:
4D33B3F7.6060306@2ndquadrant.com (view raw or flat )
Thread:
2010-11-14 23:48:24 from Greg Smith <greg(at)2ndquadrant(dot)com>
2010-11-16 02:15:32 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-20 23:21:48 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2010-11-21 01:17:47 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-21 02:11:40 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2010-11-21 16:37:26 from Greg Smith <greg(at)2ndquadrant(dot)com>
2010-11-21 23:07:20 from Josh Berkus <josh(at)agliodbs(dot)com>
2010-11-27 01:51:05 from Ron Mayer <rm_pg(at)cheapcomplexdevices(dot)com>
2010-11-30 20:29:57 from Greg Smith <greg(at)2ndquadrant(dot)com>
2010-11-30 20:58:09 from Josh Berkus <josh(at)agliodbs(dot)com>
2011-01-12 01:27:36 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-15 10:47:24 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-15 12:05:57 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-15 14:25:40 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-15 14:40:59 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-15 15:31:05 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-15 21:28:17 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-15 22:57:02 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-16 04:14:34 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-16 07:28:58 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-17 16:19:20 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-01-18 01:46:21 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-18 10:26:51 from Cédric Villemain <cedric(dot)villemain(dot)debian(at)gmail(dot)com>
2011-01-16 03:35:17 from Marti Raudsepp <marti(at)juffo(dot)org>
2011-01-16 05:11:26 from Simone Aiken <saiken(at)ulfheim(dot)net>
2011-01-16 11:29:37 from Nicolas Barbier <nicolas(dot)barbier(at)gmail(dot)com>
2011-01-16 16:34:31 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-01-16 18:47:11 from Simone Aiken <saiken(at)ulfheim(dot)net>
2011-01-18 06:47:50 from Simone Aiken <saiken(at)ulfheim(dot)net>
2011-01-18 13:35:58 from Alvaro Herrera <alvherre(at)commandprompt(dot)com>
2011-01-18 15:46:18 from Simone Aiken <saiken(at)ulfheim(dot)net>
2011-01-18 16:44:26 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-18 17:16:05 from "Simone Aiken" <saiken(at)ulfheim(dot)net>
2011-01-18 21:52:43 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-18 23:49:29 from "Simone Aiken" <saiken(at)quietlyCompetent(dot)com>
2011-01-19 18:25:00 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-19 19:25:59 from Alvaro Herrera <alvherre(at)commandprompt(dot)com>
2011-01-19 19:26:02 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-01-19 19:39:29 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-19 20:10:39 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-01-19 20:19:24 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-19 22:43:56 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-01-20 21:40:09 from "Simone Aiken" <saiken(at)quietlyCompetent(dot)com>
2011-01-21 02:16:43 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-21 15:00:14 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-01-19 21:27:16 from "Simone Aiken" <saiken(at)ulfheim(dot)net>
2011-01-20 14:27:38 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-18 21:36:25 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-01-19 16:19:47 from "Simone Aiken" <saiken(at)quietlyCompetent(dot)com>
2011-01-18 13:39:05 from Alvaro Herrera <alvherre(at)commandprompt(dot)com>
2011-01-18 11:19:15 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-15 13:55:21 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2011-01-15 14:15:49 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-15 20:53:18 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2011-01-17 23:07:38 from Jim Nasby <jim(at)nasby(dot)net>
2011-01-18 00:27:34 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-18 00:33:07 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-02-03 22:55:10 from Michael Banck <mbanck(at)debian(dot)org>
2011-02-04 18:43:23 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-17 00:32:55 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2011-01-17 01:42:13 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-17 03:13:59 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-17 03:37:33 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-17 16:40:31 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2011-01-17 18:48:54 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-18 19:12:53 from Josh Berkus <josh(at)agliodbs(dot)com>
2011-01-27 17:18:37 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-27 17:27:44 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-27 19:33:21 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-28 05:53:24 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-29 13:13:04 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-31 04:41:36 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-31 08:04:22 from Itagaki Takahiro <itagaki(dot)takahiro(at)gmail(dot)com>
2011-01-31 14:44:58 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-31 14:52:27 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2011-01-31 16:29:56 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-01-31 16:43:01 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-31 16:51:13 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-01-31 16:55:38 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-31 17:11:24 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-01-31 18:44:38 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-31 20:27:25 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-01-31 21:28:01 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-02-01 17:44:05 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-02-01 17:58:52 from "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
2011-02-01 18:32:22 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-02-01 18:32:28 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-02-01 18:30:12 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-02-01 18:35:13 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-02-01 18:48:42 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-01-31 21:04:13 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-01-31 17:01:48 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2011-01-31 17:04:00 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-31 21:33:18 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-02-01 15:49:03 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-02-01 18:30:53 from Bruce Momjian <bruce(at)momjian(dot)us>
2011-02-04 19:08:07 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-02-04 19:39:06 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-02-07 07:07:41 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-02-07 15:22:15 from Cédric Villemain <cedric(dot)villemain(dot)debian(at)gmail(dot)com>
2011-02-07 15:44:05 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-02-07 18:38:34 from "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
2011-02-07 22:06:46 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-02-11 03:30:44 from Greg Smith <greg(at)2ndquadrant(dot)com>
2011-02-11 04:01:33 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-21 21:54:00 from Greg Smith <greg(at)2ndquadrant(dot)com>
2010-11-21 22:19:30 from Martijn van Oosterhout <kleptog(at)svana(dot)org>
2010-11-21 22:45:50 from Andres Freund <andres(at)anarazel(dot)de>
2010-11-24 02:23:49 from Cédric Villemain <cedric(dot)villemain(dot)debian(at)gmail(dot)com>
2010-11-22 00:05:06 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 22:56:46 from Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
2010-12-01 04:25:47 from Greg Smith <greg(at)2ndquadrant(dot)com>
2010-12-01 08:50:14 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-12-01 21:30:07 from Greg Smith <greg(at)2ndquadrant(dot)com>
2010-12-02 06:11:21 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-12-05 21:53:41 from Greg Smith <greg(at)2ndquadrant(dot)com>
2010-12-05 22:32:28 from Rob Wultsch <wultsch(at)gmail(dot)com>
2010-12-05 23:02:48 from Greg Smith <greg(at)2ndquadrant(dot)com>
2010-12-07 02:26:28 from Alvaro Herrera <alvherre(at)commandprompt(dot)com>
2010-12-07 15:27:28 from Greg Smith <greg(at)2ndquadrant(dot)com>
2010-12-08 15:22:59 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2010-12-02 19:24:13 from Greg Stark <gsstark(at)mit(dot)edu>
2010-12-02 19:48:54 from Josh Berkus <josh(at)agliodbs(dot)com>
2010-12-03 00:12:16 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-05 05:56:51 from Greg Smith <greg(at)2ndquadrant(dot)com>
Lists:
pgsql-hackers
I have finished a first run of benchmarking the current 9.1 code at
various sizes. See http://www.2ndquadrant.us/pgbench-results/index.htm
for many details. The interesting stuff is in Test Set 3, near the
bottom. That's the first one that includes buffer_backend_fsync data.
This iall on ext3 so far, but is using a newer 2.6.32 kernel, the one
from Ubuntu 10.04.
The results are classic Linux in 2010: latency pauses from checkpoint
sync will easily leave the system at a dead halt for a minute, with the
worst one observed this time dropping still for 108 seconds. That one
is weird, but these two are completely averge cases:
http://www.2ndquadrant.us/pgbench-results/210/index.html
http://www.2ndquadrant.us/pgbench-results/215/index.html
I think a helpful next step here would be to put Robert's fsync
compaction patch into here and see if that helps. There are enough
backend syncs showing up in the difficult workloads (scale>=1000,
clients >=32) that its impact should be obvious.
--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books
In response to
Responses
pgsql-hackers by date
Next :From: Josh KupershmidtDate: 2011-01-17 03:32:51Subject : Re: psql: Add \dL to show languagesPrevious :From : Fujii MasaoDate : 2011-01-17 03:05:22Subject : Re: walreceiver fallback_application_name
