Re: COPY enhancements

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, Greg Smith <gsmith(at)gregsmith(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: COPY enhancements
Date: 2009-09-14 00:17:45
Message-ID: 4AAD8BA9.6050800@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tom Lane wrote:
> Josh Berkus <josh(at)agliodbs(dot)com> writes:
>
>> It's not as if we don't have the ability to measure performance impact.
>> It's reasonable to make a requirement that new options to COPY
>> shouldn't slow it down noticeably if those options aren't used. And we
>> can test that, and even make such testing part of the patch review.
>>
>
> Really? Where is your agreed-on, demonstrated-to-be-reproducible
> benchmark for COPY speed?
>
> My experience is that reliably measuring performance costs in the
> percent-or-so range is *hard*. It's only after you've added a few of
> them and they start to mount up that it becomes obvious that all those
> insignificant additions really did cost something.
>

Well, I strongly suspect that the cost of the patch I'm working with is
not in the percent-or-so range, and much more likely to be in the
tiny-fraction-of-a-percent range. The total overhead in the non-ragged
case is one extra test per field, plus one per null field, plus two
tests per line.

But since you raise the question I'll conduct some tests and then you
can criticize those. Ruling out tests a priori seems a bit extreme.

The current patch is attached for information (and in case anyone else
wants to try some testing).

cheers

andrew

Attachment Content-Type Size
raggedcopy.patch text/x-patch 8.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Itagaki Takahiro 2009-09-14 00:43:41 syslog_line_prefix
Previous Message Itagaki Takahiro 2009-09-14 00:16:01 Re: Why does LOG have higher priority than ERROR and WARNING?