Re: COPY enhancements

From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: <jd(at)commandprompt(dot)com>,"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Emmanuel Cecchet" <Emmanuel(dot)Cecchet(at)asterdata(dot)com>, "Emmanuel Cecchet" <manu(at)asterdata(dot)com>, "Robert Haas" <robertmhaas(at)gmail(dot)com>, "Dimitri Fontaine" <dfontaine(at)hi-media(dot)com>, "PostgreSQL-development" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: COPY enhancements
Date: 2009-10-08 16:49:33
Message-ID: 4ACDD1CD020000250002B751@gw.wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Hmm, if we were willing to break COPY into multiple *top level*
> transactions, that would avoid my concern about XID wraparound.
> The issue here is that if the COPY does eventually fail (and there
> will always be failure conditions, eg out of disk space), then some
> of the previously entered rows would still be there; but possibly
> not all of them, depending on whether we batch rows. The latter
> property actually bothers me more than the former, because it would
> expose an implementation detail to the user. Thoughts?
>
> Also, this does not work if you want the copy to be part of a bigger
> transaction, viz
> BEGIN;
> do something;
> COPY ...;
> do something else;
> COMMIT;

I was considering suggesting multiple top-level transactions, partly
based on the fact that other bulk-load utilities that I've used
support that. Then I realized that those were *client* side
applications, and didn't have to worry about being part of an
enclosing transaction. It seems wrong to break transactional
semantics in a single statement execution. Multiple top-level
transactions could be fine in a bulk-load *application*; but not, in
my view, in the COPY *statement*.

-Kevin

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2009-10-08 16:50:41 Re: COPY enhancements
Previous Message Jeff Davis 2009-10-08 16:47:34 Re: Issues for named/mixed function notation patch