Re: [HACKERS] PATCH: Batch/pipelining support for libpq

From: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Justin Pryzby <pryzby(at)telsasoft(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Craig Ringer <craig(dot)ringer(at)enterprisedb(dot)com>, Matthieu Garrigues <matthieu(dot)garrigues(at)gmail(dot)com>, Zhihong Yu <zyu(at)yugabyte(dot)com>, Álvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Aya Iwata <iwata(dot)aya(at)jp(dot)fujitsu(dot)com>, Daniel Vérité <daniel(at)manitou-mail(dot)org>, "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>, Kirk Jamison <k(dot)jamison(at)fujitsu(dot)com>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, Nikhil Sontakke <nikhils(at)2ndquadrant(dot)com>, Vaishnavi Prabakaran <VaishnaviP(at)fast(dot)au(dot)fujitsu(dot)com>
Subject: Re: [HACKERS] PATCH: Batch/pipelining support for libpq
Date: 2021-03-15 18:54:54
Message-ID: 20210315185454.GA2830@alvherre.pgsql
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Here's what seems a final version of the patch. I renamed one more
function: PQsendPipeline is now PQpipelineSync. I also reworded the
docs in a couple of places, added a few tests to the pgbench patch, and
made it work.

Note the pgbench results in pipeline mode:

./pgbench -r -Mextended -n -f /home/alvherre/Code/pgsql-build/pipeline/src/bin/pgbench/tmp_check/t_001_pgbench_with_server_main_data/001_pgbench_pipeline -c 100 -t10000
pgbench (PostgreSQL) 14.0
transaction type: /home/alvherre/Code/pgsql-build/pipeline/src/bin/pgbench/tmp_check/t_001_pgbench_with_server_main_data/001_pgbench_pipeline
scaling factor: 1
query mode: extended
number of clients: 100
number of threads: 1
number of transactions per client: 10000
number of transactions actually processed: 1000000/1000000
latency average = 2.316 ms
initial connection time = 113.859 ms
tps = 43182.438635 (without initial connection time)
statement latencies in milliseconds:
0.000 \startpipeline
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
0.000 select 1;
1.624 \endpipeline

If I just replace the \startpipeline and \endpipeline lines with BEGIN
and COMMIT respectively, I get this:

tps = 10220.259051 (without initial connection time)

0.830 begin;
0.765 select 1;
0.752 select 1;
0.753 select 1;
0.755 select 1;
0.754 select 1;
0.755 select 1;
0.757 select 1;
0.756 select 1;
0.756 select 1;
0.756 select 1;
0.750 commit;

Yes, you could say that this is a liiiitle bit unfair -- but it seems
quite impressive nonetheless.

--
Álvaro Herrera 39°49'30"S 73°17'W

Attachment Content-Type Size
v37-0001-Implement-pipeline-mode-in-libpq.patch text/x-diff 113.0 KB
v37-0002-Add-libpq-pipeline-mode-support-to-pgbench.patch text/x-diff 9.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2021-03-15 18:57:20 Re: pg_amcheck contrib application
Previous Message Tom Lane 2021-03-15 18:49:51 Re: Tightening up allowed custom GUC names