Re: libpq: PQgetCopyData() and allocation overhead

Update: in the latest iteration, I have an alternative that reduces CPU
time by more than half, compared to PQgetCopyData().

And the code is simpler, too, both in the client and in libpq itself. The
one downside is that it breaks with libpq's existing API style.

PR for easy discussion: https://github.com/jtv/postgres/pull/1

Jeroen

On Mon, 27 Feb 2023 at 17:08, Jeroen Vermeulen <jtvjtv(at)gmail(dot)com> wrote:

> Done. Thanks for looking!
>
> Jelte Fennema pointed out that I should probably be using PQExpBuffer for
> this. I'll look into that later; this is a proof of concept, not a
> production-ready API proposal.
>
>
> Jeroen
>
> On Mon, 27 Feb 2023 at 14:48, Bharath Rupireddy <
> bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
>
>> On Fri, Feb 10, 2023 at 5:49 PM Jeroen Vermeulen <jtvjtv(at)gmail(dot)com>
>> wrote:
>> >
>> > OK, I've updated the PR with a benchmark (in the main directory).
>> >
>> > On this benchmark I'm seeing about a 24% reduction in "user" CPU time,
>> and a 8% reduction in "system" CPU time. (Almost no reduction in
>> wall-clock time.)
>>
>> I can help run some logical replication performance benchmarks
>> tomorrow. Would you mind cleaning the PR and providing the changes
>> (there are multiple commits in the PR) as a single patch here for the
>> sake of ease of review and test?
>>
>> --
>> Bharath Rupireddy
>> PostgreSQL Contributors Team
>> RDS Open Source Databases
>> Amazon Web Services: https://aws.amazon.com
>>
>