Skip site navigation (1) Skip section navigation (2)

Re: Best COPY Performance

From: "Craig A(dot) James" <cjames(at)modgraph-usa(dot)com>
To: "Jim C(dot) Nasby" <jim(at)nasby(dot)net>
Cc: Worky Workerson <worky(dot)workerson(at)gmail(dot)com>, Merlin Moncure <mmoncure(at)gmail(dot)com>, pgsql-performance(at)postgresql(dot)org
Subject: Re: Best COPY Performance
Date: 2006-10-23 22:37:47
Message-ID: 453D443B.20302@modgraph-usa.com (view raw or flat)
Thread:
Lists: pgsql-performance
Jim C. Nasby wrote:
> http://stats.distributed.net used to use a perl script to do some
> transformations before loading data into the database. IIRC, when we
> switched to using C we saw 100x improvement in speed, so I suspect that
> if you want performance perl isn't the way to go. I think you can
> compile perl into C, so maybe that would help some.

I use Perl extensively, and have never seen a performance problem.  I suspect the perl-to-C "100x improvement" was due to some other factor, like a slight change in the schema, indexes, or the fundamental way the client (C vs Perl) handled the data during the transformation, or just plain bad Perl code.

Modern scripting languages like Perl and Python make programmers far, far more productive than the bad old days of C/C++.  Don't shoot yourself in the foot by reverting to low-level languages like C/C++ until you've exhausted all other possibilities.  I only use C/C++ for intricate scientific algorithms.

In many cases, Perl is *faster* than C/C++ code that I write, because I can't take the time (for example) to write the high-performance string manipulation that have been fine-tuned and extensively optimized in Perl.

Craig


In response to

Responses

pgsql-performance by date

Next:From: Péter KovácsDate: 2006-10-23 23:07:35
Subject: Re: Index on two columns not used
Previous:From: Joshua D. DrakeDate: 2006-10-23 22:10:33
Subject: Re: Best COPY Performance

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group