Re: [patch] libpq one-row-at-a-time API

From: Marko Kreen <markokr(at)gmail(dot)com>
To: Postgres Hackers <pgsql-hackers(at)postgresql(dot)org>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Merlin Moncure <mmoncure(at)gmail(dot)com>
Subject: Re: [patch] libpq one-row-at-a-time API
Date: 2012-07-24 20:28:53
Message-ID: 20120724202853.GA3974@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> Hm, I think it's possible to rig the test to do dummy
> copy of pgresult, thus it's possible to see what kind
> of speed is possible.. Will try.

I added a new method (-x) to rowdump where it asks for row
with PQgetRowData() and then tries to emulate super-efficient
PGresult copy, then loads data from that PGresult.

Quick reference:
rowdump1 - single-row-mode1 [~ libpq 9.2]
rowdump2 - single-row-mode2 [~ libpq 9.1]

-s - single row mode with PQgetResult()
-z - single row mode with PQgetRowData()
-x - simulated optimized PQgetResult()

-------------------------------------------------------------
QUERY: select 10000,200,300000,rpad('x',30,'z') from
generate_series(1,5000000)
./rowdump1 -s: 6.28 6.25 6.39 avg: 6.31 [ 100.00 % ]
./rowdump2 -s: 7.49 7.40 7.57 avg: 7.49 [ 118.71 % ]
./rowdump1 -z: 2.86 2.77 2.79 avg: 2.81 [ 44.50 % ]
./rowdump1 -x: 3.46 3.27 3.29 avg: 3.34 [ 52.96 % ]
QUERY: select
rpad('x',10,'z'),rpad('x',20,'z'),rpad('x',30,'z'),rpad('x',40,'z'),rpad('x',50,'z'),rpad('x',60,'z')
from generate_series(1,3000000)
./rowdump1 -s: 7.76 7.76 7.68 avg: 7.73 [ 100.00 % ]
./rowdump2 -s: 8.24 8.12 8.66 avg: 8.34 [ 107.85 % ]
./rowdump1 -z: 5.34 5.07 5.23 avg: 5.21 [ 67.41 % ]
./rowdump1 -x: 5.53 5.61 5.61 avg: 5.58 [ 72.20 % ]
QUERY: select
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100
from generate_series(1,800000)
./rowdump1 -s: 7.49 7.66 7.59 avg: 7.58 [ 100.00 % ]
./rowdump2 -s: 7.56 8.12 7.95 avg: 7.88 [ 103.91 % ]
./rowdump1 -z: 2.77 2.76 2.76 avg: 2.76 [ 36.46 % ]
./rowdump1 -x: 3.07 3.05 3.18 avg: 3.10 [ 40.90 % ]
QUERY: select 1000,rpad('x', 400, 'z'),rpad('x', 4000, 'z') from
generate_series(1,100000)
./rowdump1 -s: 2.66 2.62 2.67 avg: 2.65 [ 100.00 % ]
./rowdump2 -s: 3.11 3.14 3.11 avg: 3.12 [ 117.74 % ]
./rowdump1 -z: 2.49 2.46 2.47 avg: 2.47 [ 93.33 % ]
./rowdump1 -x: 2.59 2.57 2.57 avg: 2.58 [ 97.23 % ]
-----------------------------------------------------------------

It shows that even if the actual "fast" row copy will be slower
than this one here, it's still quote competitive approach to
PQgetRowData(), as long it's not too slow.

So the optimized PGgetResult() may be good enough, thus we
can drop the idea of PQgetRowData().

Code attached, also in https://github.com/markokr/pqtest repo.

--
marko

Attachment Content-Type Size
rowdump.c text/x-csrc 8.1 KB
xtest.sh application/x-sh 1.2 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2012-07-24 20:31:02 Re: canceling autovacuum task woes
Previous Message Robert Haas 2012-07-24 20:14:01 Re: canceling autovacuum task woes