Skip site navigation (1) Skip section navigation (2)

Query ResultSet parsing speedup patch (resend)

From: Mikko Tiihonen <mikko(dot)tiihonen(at)iki(dot)fi>
To: pgsql-jdbc(at)postgresql(dot)org
Subject: Query ResultSet parsing speedup patch (resend)
Date: 2006-09-29 22:01:13
Message-ID: 1159567273.19055.7.camel@dual.local (view raw or flat)
Thread:
Lists: pgsql-jdbc
Hi,

This is a resend since I did not get any feedback to this patch when I
sent it two months ago. Could someone at least have some recommendations
what I should do to get this patch evaluated or even applied to the
official source.

Also, could someone update the CVS info page
(http://jdbc.postgresql.org/development/cvs.html) to point to the new
pgfoundry ?

---

I tried to optimise the parsing of ResultSets from the network stream,
especially int and long values, but also a bit for String values.

The attached patch gives upto 40% speedup when parsing larger queries
containing equal amounts of int4, int8 or varchar(16) columns if the
system is cpu bound. The speedup comes mainly from avoiding creation of
useless objects and avoiding useless byte[] copies. In more real-life
scenarios the speedup will be much smaller.

The patch also contains the test code which I used to test the
performance (BenchTest.java). The benchmark results are:

unpatched 503 jdbc driver: speed: 180.18  memory: 36.8MB
patched 503 jdbc driver:   speed: 261.44  memory: 19.6MB
                                  +40%            -40%
The benchmark was run on Java6rc build 100 with postgresql 8.1.4 running
on localhost with Athlon64 2x2GHz, 64bit mode.

If the patch is accepted I can try to do similar optimisations for other
fields.

---

What the patch does:
Parsing string:
old way: read byte[] containing the exact bytes of a string
new way: create a string directly from network buffer with start and end
index

Parsing numbers:
old way: create a string that contains the number and use
Integer.parseInt
new way: parse the number directly from bytes

Exact changes explained:

new class VisibleBufferedInputStream:
- replaced java.io.BufferedInputStream with faster implementation
  * no synchronisation
  * allows direct access to the buffer byte[] contents which helps
    to avoiding useless copies when converting to String
  * has method for scanning the length of next null terminated string

PGStream:
- uses VisibleBuffereInputStream
  * faster implementations for for ReceiveIntegerR and ReceiveString

AbstractJdbc2ResultSet
-  parses int and long values directly from byte[] -> number instead
   of first creating a throw-away string
-  if the fast conversion fails or if the current charset is not safe
   for the optimisation then uses the old way to parse
-  also very minor optimisation to getFixedString to optimise for
   non-money string types
   
Encoding:
-  uses HashMap instead of synchronised Hashtable
-  does not do two queries to encodings map when obtaining database
   encoding
-  added method to check if the numbers match ASCII charset locations
   (all currently listed charsets match)

Attachment: postgresql-resultset-parsing-speedup.patch
Description: text/x-patch (25.6 KB)

Responses

pgsql-jdbc by date

Next:From: Michael PaesoldDate: 2006-09-30 18:47:43
Subject: Re: [pgsql-jdbc] dollar-quoted CREATE FUNCTION statement fails
Previous:From: Sriram DandapaniDate: 2006-09-29 21:44:15
Subject: Re: [JDBC] number of transactions doubling

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group