From:
"Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To:
"Robert Haas" <robertmhaas(at)gmail(dot)com>, "Jeff Davis" <pgsql(at)j-davis(dot)com>
Cc:
"Heikki Linnakangas" <heikki(dot)linnakangas(at)enterprisedb(dot)com>, "Bruce Momjian" <bruce(at)momjian(dot)us>, <pgsql-hackers(at)postgresql(dot)org>,"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Subject:
Re: crash-safe visibility map, take three
Date:
2010-12-02 19:16:09
Message-ID:
4CF79C19020000250003819A@gw.wicourts.gov (view raw or flat )
Thread:
2010-11-30 04:57:25 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 07:34:04 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-11-30 15:32:15 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 15:43:47 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-11-30 15:54:50 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 16:40:10 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 16:48:08 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-11-30 16:54:43 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 16:59:20 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 17:01:56 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 15:38:02 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 15:46:44 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 15:53:25 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-11-30 16:02:05 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-11-30 16:16:33 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 16:22:03 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 16:27:04 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 16:33:18 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 16:47:40 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 16:49:34 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-11-30 16:55:44 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 17:00:29 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 17:10:49 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 17:12:35 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 17:22:02 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 17:25:18 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-11-30 17:36:07 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 17:25:41 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 17:26:08 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-11-30 16:59:47 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-01 01:35:56 from Bruce Momjian <bruce(at)momjian(dot)us>
2010-12-01 07:35:40 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-12-01 13:39:07 from Bruce Momjian <bruce(at)momjian(dot)us>
2010-12-01 13:53:12 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-12-01 14:57:51 from "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
2010-12-01 15:46:21 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-01 15:36:51 from Bruce Momjian <bruce(at)momjian(dot)us>
2010-12-01 16:25:36 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-01 16:40:39 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-12-01 16:43:39 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-12-01 17:23:50 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-01 16:40:46 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-12-01 16:44:12 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-12-01 16:48:03 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-12-01 17:22:41 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-12-01 17:31:36 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-01 20:31:27 from Jeff Davis <pgsql(at)j-davis(dot)com>
2010-12-01 20:59:49 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-01 21:27:28 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-12-01 22:24:22 from Jeff Davis <pgsql(at)j-davis(dot)com>
2010-12-02 04:22:12 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-02 11:37:57 from Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
2010-12-02 14:37:37 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-02 19:01:43 from Jeff Davis <pgsql(at)j-davis(dot)com>
2010-12-02 19:16:09 from "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
2010-12-02 22:00:18 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-02 23:37:05 from Jeff Davis <pgsql(at)j-davis(dot)com>
2010-12-03 00:06:29 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-03 00:40:44 from Jeff Davis <pgsql(at)j-davis(dot)com>
2010-12-01 23:41:33 from Jim Nasby <jim(at)nasby(dot)net>
2010-12-02 04:36:42 from Robert Haas <robertmhaas(at)gmail(dot)com>
2010-12-03 14:25:20 from Florian Weimer <fweimer(at)bfk(dot)de>
2010-11-30 16:27:30 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-11-30 16:10:13 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 16:30:53 from Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
2010-11-30 16:37:13 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2010-11-30 13:18:12 from Rob Wultsch <wultsch(at)gmail(dot)com>
2011-01-05 20:22:54 from Jesper Krogh <jesper(at)krogh(dot)cc>
2011-01-06 02:10:15 from Robert Haas <robertmhaas(at)gmail(dot)com>
2011-01-06 07:00:29 from Jesper Krogh <jesper(at)krogh(dot)cc>
2011-01-07 18:28:54 from Jim Nasby <jim(at)nasby(dot)net>
2011-01-08 02:59:36 from Robert Haas <robertmhaas(at)gmail(dot)com>
Lists:
pgsql-hackers
Jeff Davis <pgsql(at)j-davis(dot)com> wrote:
> And, if we had a bulk loading path, we could probably get away
> with writing the data only twice (today, we write it 3 times
> including the hint bits) or maybe once if WAL archiving is off.
If you're counting WAL writes, you're low. If you don't go out of
your way to avoid it, you are likely to write the data to the table
once during the bulk load, a second time on first read to set the
hint bits, and a third time to freeze data to prevent wrap-around.
The initial write may or may not be WAL-logged. The freezing
typically is WAL-logged.
So, you can easily write the data to disk four or five times. With
luck these are spread out uniformly or happen during off-peak
periods. Unmanaged, a WAL-logged freeze of bulk-loaded data is
somewhat more likely to occur, however, during hours of heavy OLTP
load, because transaction numbers are consumed so quickly.
Currently, a VACUUM FREEZE after a bulk load collapses at least two
of those writes to one. With luck, some pages might still be dirty
in cache, and you can save two of the writes.
-Kevin
In response to
pgsql-hackers by date
Next :From: Greg StarkDate: 2010-12-02 19:24:13
Subject : Re: Spread checkpoint sync
Previous :From : Joachim WielandDate : 2010-12-02 19:13:13
Subject : Re: WIP patch for parallel pg_dump