Re: COPY enhancements

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Josh Berkus <josh(at)agliodbs(dot)com>
Cc: Andrew Dunstan <andrew(at)dunslane(dot)net>, Greg Smith <gsmith(at)gregsmith(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: COPY enhancements
Date: 2009-09-13 21:33:59
Message-ID: 20524.1252877639@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> It's not as if we don't have the ability to measure performance impact.
> It's reasonable to make a requirement that new options to COPY
> shouldn't slow it down noticeably if those options aren't used. And we
> can test that, and even make such testing part of the patch review.

Really? Where is your agreed-on, demonstrated-to-be-reproducible
benchmark for COPY speed?

My experience is that reliably measuring performance costs in the
percent-or-so range is *hard*. It's only after you've added a few of
them and they start to mount up that it becomes obvious that all those
insignificant additions really did cost something.

But in any case, I think that having a clear distinction between
"straight data import" and "data transformation" features is a good
thing. COPY is already pretty much of an unmanageable monstrosity,
and continuing to accrete features into it without any sort of structure
is something we are going to regret.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message decibel 2009-09-13 21:47:49 Re: RfD: more powerful "any" types
Previous Message Josh Berkus 2009-09-13 21:17:51 Re: COPY enhancements