Re: performance for high-volume log insertion

On Wed, Apr 22, 2009 at 4:37 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> Thanks for doing the work.  I had been intending to but hadn't gotten to
> it yet.

I'd done similar tests recently, for some batch import code, so it was
just a matter of recreating it.

>> separate inserts, no transaction: 21.21s
>> separate inserts, same transaction: 1.89s
>> 40 inserts, 100 rows/insert: 0.18s
>> one 40000-value insert: 0.16s
>> 40 prepared inserts, 100 rows/insert: 0.15s
>> COPY (text): 0.10s
>> COPY (binary): 0.10s
>
> What about 40000 individual prepared inserts?  Just curious about it.

40000 inserts, one prepared statement each (constructing the prepared
statement only once), in a single transaction: 1.68s

I'm surprised that there's any win here at all.

> Also, kind of suprised about COPY text vs. binary.  What was the data
> type in the table..?  If text, that makes sense, if it was an integer or
> something else, I'm kind of suprised.

Each row had one integer column. I expect strings to be harder to
parse, since it's allocating buffers and parsing escapes, which is
usually more expensive than parsing an integer out of a string. I'd
expect the difference to be negligible either way, though, and I'd be
interested in hearing about use cases where binary copying is enough
of a win to be worth the development cost of maintaining the feature.

On Wed, Apr 22, 2009 at 4:49 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Also, just to be clear: were the "40 insert" cases 40 separate
> transactions or one transaction? I think the latter was meant but
> it's not 100% clear.

One transaction--the only case where I ran more than one transaction
was the first.

--
Glenn Maynard