Re: Speed dblink using alternate libpq tuple storage

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)oss(dot)ntt(dot)co(dot)jp>
To: pgsql-hackers(at)postgresql(dot)org, greg(at)2ndQuadrant(dot)com
Subject: Re: Speed dblink using alternate libpq tuple storage
Date: 2012-01-17 08:53:33
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hello, This is revised and rebased version of the patch.

a. Old term `Add Tuple Function' is changed to 'Store
Handler'. The reason why not `storage' is simply length of the

b. I couldn't find the place to settle PGgetAsCString() in. It is
removed and storeHandler()@dblink.c touches PGresAttValue
directly in this new patch. Definition of PGresAttValue stays
in lipq-fe.h and provided with comment.

c. Refine error handling of dblink.c. I think it preserves the
previous behavior for column number mismatch and type
conversion exception.

d. Document is revised.

> It jumped from 332K tuples/sec to 450K, a 35% gain, and had a
> lower memory footprint too. Test methodology and those results
> are at

It is a disappointment that I found that the gain had become
lower than that according to the re-measuring.

For CentOS6.2 and other conditions are the same to the previous
testing, the overall performance became hihger and the loss of
libpq patch was 1.8% and the gain of full patch had been fallen
to 5.6%. But the reduction of the memory usage was not changed.

Original : 3.96s 100.0%
w/libpq patch : 4.03s 101.8%
w/libpq+dblink patch : 3.74s 94.4%

The attachments are listed below.

- Allow alternative storage for libpql.

- Modify dblink to use alternative storage mechanism.

- Document for libpq_altstore. Shows in "31.19. Alternatie result storage"


Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
libpq_altstore_20120117.patch text/x-patch 15.6 KB
dblink_perf_20120117.patch text/x-patch 11.8 KB
libpq_altstore_doc_20120117.patch text/x-patch 9.2 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Joel Jacobson 2012-01-17 09:13:53 Re: Generate call graphs in run-time
Previous Message Alexander Korotkov 2012-01-17 08:04:06 Re: Collect frequency statistics for arrays