From:
Simon Riggs <simon(at)2ndquadrant(dot)com>
To:
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc:
Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>,Greg Stark <gsstark(at)mit(dot)edu>, Neil Conway <neilc(at)samurai(dot)com>,Alvaro Herrera <alvherre(at)surnet(dot)cl>, pgsql-hackers(at)postgresql(dot)org,Alon Goldshuv <agoldshuv(at)greenplum(dot)com>
Subject:
Re: NOLOGGING option, or ?
Date:
2005-06-01 20:33:24
Message-ID:
1117658004.3844.1059.camel@localhost.localdomain (view raw or flat )
Thread:
2005-05-31 23:01:43 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 02:47:30 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-01 04:40:07 from Alvaro Herrera <alvherre(at)surnet(dot)cl>
2005-06-01 05:00:04 from Neil Conway <neilc(at)samurai(dot)com>
2005-06-01 08:21:39 from Greg Stark <gsstark(at)mit(dot)edu>
2005-06-01 08:44:24 from Greg Stark <gsstark(at)mit(dot)edu>
2005-06-01 12:03:25 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 13:39:07 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-01 13:58:53 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-01 14:12:18 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-01 14:18:25 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-01 15:21:07 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 15:55:46 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-01 17:55:46 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 18:10:45 from Alvaro Herrera <alvherre(at)surnet(dot)cl>
2005-06-01 18:24:39 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-01 20:17:39 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 18:14:27 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-01 20:33:24 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 22:05:13 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-01 22:32:32 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-01 22:47:17 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 23:02:17 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-01 22:37:51 from Hannu Krosing <hannu(at)tm(dot)ee>
2005-06-01 14:07:37 from Jochem van Dieten <jochemd(at)gmail(dot)com>
2005-06-01 14:14:42 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-01 15:43:24 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-01 16:00:28 from Jochem van Dieten <jochemd(at)gmail(dot)com>
2005-06-01 21:27:01 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-01 08:16:46 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 08:34:38 from Christopher Kings-Lynne <chriskl(at)familyhealth(dot)com(dot)au>
2005-06-01 11:54:19 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 14:01:06 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-01 09:54:58 from Hannu Krosing <hannu(at)skype(dot)net>
2005-06-01 17:35:30 from "Alon Goldshuv" <agoldshuv(at)greenplum(dot)com>
2005-06-01 21:56:07 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-01 22:18:41 from "Luke Lonergan" <llonergan(at)greenplum(dot)com>
2005-06-01 22:37:07 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-02 00:05:29 from "Luke Lonergan" <llonergan(at)greenplum(dot)com>
2005-06-02 01:05:11 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-02 01:54:51 from "Luke Lonergan" <llonergan(at)greenplum(dot)com>
2005-06-02 02:27:51 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-02 02:40:06 from "Luke Lonergan" <llonergan(at)greenplum(dot)com>
2005-06-01 23:00:34 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-01 23:29:30 from "Alon Goldshuv" <agoldshuv(at)greenplum(dot)com>
2005-06-02 01:01:56 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-02 01:50:23 from "Luke Lonergan" <llonergan(at)greenplum(dot)com>
2005-06-02 02:13:38 from Oliver Jowett <oliver(at)opencloud(dot)com>
2005-06-02 02:28:41 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-02 02:35:33 from "Luke Lonergan" <llonergan(at)greenplum(dot)com>
2005-06-02 02:47:26 from Steve Atkins <steve(at)blighty(dot)com>
2005-06-02 03:35:57 from "Luke Lonergan" <llonergan(at)greenplum(dot)com>
2005-06-02 05:16:34 from "Luke Lonergan" <llonergan(at)greenplum(dot)com>
2005-06-01 22:39:59 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 23:06:07 from Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
2005-06-01 23:06:09 from Joe Conway <mail(at)joeconway(dot)com>
2005-06-02 03:41:29 from Alvaro Herrera <alvherre(at)surnet(dot)cl>
2005-06-02 16:54:56 from "Alon Goldshuv" <agoldshuv(at)greenplum(dot)com>
2005-06-01 06:25:42 from Hans-Jürgen Schönig <postgres(at)cybertec(dot)at>
2005-06-01 14:43:44 from Dawid Kuroczko <qnex42(at)gmail(dot)com>
2005-06-01 12:10:55 from Hannu Krosing <hannu(at)skype(dot)net>
2005-06-01 15:31:31 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2005-06-01 16:18:09 from Hannu Krosing <hannu(at)tm(dot)ee>
2005-06-01 17:00:28 from Simon Riggs <simon(at)2ndquadrant(dot)com>
2005-06-01 17:18:55 from Alvaro Herrera <alvherre(at)surnet(dot)cl>
2005-06-01 17:35:25 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Lists:
pgsql-hackers
On Wed, 2005-06-01 at 14:14 -0400, Tom Lane wrote:
> So what I'm thinking is we need no special WAL entries for this. What
> we need is just an operating mode of COPY in which it doesn't WAL-log
> its inserts, but instead fsyncs before completion, much like index build
> does. For safety it must do all its inserts into freshly-added pages;
> this is not to ensure truncatability, because we aren't going to do that
> anyway, but to ensure that we don't have unlogged operations changing
> pages that might contain committed tuples. (That would pose a risk of
> losing committed data to incomplete writes in case of system crash
> partway through. The same reason is why we need exclusive lock: else
> we might end up with pages containing a mix of logged and unlogged
> tuples.) Also there can be no indexes, since we don't want index
> entries pointing to unlogged tuples. And PITR can't be enabled.
> Otherwise no problem.
What you describe above is a coherent set of features that provide most
of the benefits I sought, plus some others. We also don't mess with WAL,
which is grand thing. We gain the ability to load into tables with rows
already in them.
I don't agree with all of your other points, but given time schedules, I
think that we win with the above, so forget the rest.
The main COPY/LOAD DATA discussion is on another thread of this from
Alon, who has some interesting ideas and some really cool performance
results to share.
Best Regards, Simon Riggs
In response to
pgsql-hackers by date
Next :From: Meredith L. PattersonDate: 2005-06-01 20:50:33
Subject : Re: Google's Summer of Code ...
Previous :From : Oleg BartunovDate : 2005-06-01 20:29:57
Subject : Re: Google's Summer of Code ...