Skip site navigation (1) Skip section navigation (2)

Re: Help with bulk read performance

From: Nick Matheson <Nick(dot)D(dot)Matheson(at)noaa(dot)gov>
To: Pierre C <lists(at)peufeu(dot)com>
Cc: Andy Colson <andy(at)squeakycode(dot)net>, Dan Schaffer <Daniel(dot)S(dot)Schaffer(at)noaa(dot)gov>, Jim Nasby <jim(at)nasby(dot)net>, pgsql-performance(at)postgresql(dot)org
Subject: Re: Help with bulk read performance
Date: 2010-12-17 13:51:27
Message-ID: 4D0B6ADF.7050303@noaa.gov (view raw or flat)
Thread:
Lists: pgsql-performance
Pierre-

I agree with your observation of float <-> text conversion costs, but in 
this case Dan is talking about storing the raw float data (ie: 4 bytes 
per float) in a bytea array so there is only the conversion from java 
float[n] to java byte[4*n] which is not nearly as costly as float <-> 
text conversion (especially if you leave it in architecture byte order).

Nick
>
>> If the data are stored as a byte array but retrieve into a ResultSet, 
>> the unpacking time goes up by an order of magnitude and the
>> observed total throughput is 25 MB/s.  If the data are stored in a 
>> Postgres float array and unpacked into a byte stream, the
>> observed throughput is 20 MB/s.
>
>
> float <-> text conversions are very slow, this is in fact due to the 
> mismatch between base-2 (IEEE754) and base-10 (text) floating point 
> representation, which needs very very complex calculations.


In response to

pgsql-performance by date

Next:From: Marti RaudseppDate: 2010-12-17 14:01:50
Subject: Re: postgres performance tunning
Previous:From: Greg SmithDate: 2010-12-17 13:20:19
Subject: Re: postgres performance tunning

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group