Skip site navigation (1) Skip section navigation (2)

Re: COPY enhancements

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, Greg Smith <gsmith(at)gregsmith(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: COPY enhancements
Date: 2009-09-14 00:17:45
Message-ID: 4AAD8BA9.6050800@dunslane.net (view raw or flat)
Thread:
Lists: pgsql-hackers

Tom Lane wrote:
> Josh Berkus <josh(at)agliodbs(dot)com> writes:
>   
>> It's not as if we don't have the ability to measure performance impact.
>>  It's reasonable to make a requirement that new options to COPY
>> shouldn't slow it down noticeably if those options aren't used.  And we
>> can test that, and even make such testing part of the patch review.
>>     
>
> Really?  Where is your agreed-on, demonstrated-to-be-reproducible
> benchmark for COPY speed?
>
> My experience is that reliably measuring performance costs in the
> percent-or-so range is *hard*.  It's only after you've added a few of
> them and they start to mount up that it becomes obvious that all those
> insignificant additions really did cost something.
>   

Well, I strongly suspect that the cost of the patch I'm working with is 
not in the percent-or-so range, and much more likely to be in the 
tiny-fraction-of-a-percent range. The total overhead in the non-ragged 
case is one extra test per field, plus one per null field, plus two 
tests per line.

But since you raise the question I'll conduct some tests and then you 
can criticize those. Ruling out tests a priori seems a bit extreme.

The current patch is attached for information (and in case anyone else 
wants to try some testing).

cheers

andrew

Attachment: raggedcopy.patch
Description: text/x-patch (8.4 KB)

In response to

pgsql-hackers by date

Next:From: Itagaki TakahiroDate: 2009-09-14 00:43:41
Subject: syslog_line_prefix
Previous:From: Itagaki TakahiroDate: 2009-09-14 00:16:01
Subject: Re: Why does LOG have higher priority than ERROR and WARNING?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group