Re: libpq: PQgetCopyData() and allocation overhead

On Fri, 3 Mar 2023 at 17:33, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

>
> If you mean exposing PQExpBuffer to users of libpq-fe.h, I'd be very
> seriously against that. I realize that libpq exposes it at an ABI
> level, but that doesn't mean we want non-Postgres code to use it.
> I also don't see what it'd add for this particular use-case.
>

Fair enough. Never even got around to checking whether it was in the API
already.

> One thing I don't care for at all in the proposed API spec is the bit
> about how the handler function can scribble on the passed buffer.
> Let's not do that. Declare it const char *, or maybe better const void *.
>

Personally I would much prefer "char" over "void" here:
* It really is a char buffer, containing text.
* If there is to be any type punning, best have it explicit.
* Reduces risk of getting the two pointer arguments the wrong way around.

As for const, I would definitely have preferred that. But if the caller
needs a zero-terminated string, forcing them into a memcpy() would kind of
defeat the purpose.

I even tried poking a terminating zero in there from inside the function,
but that made the code significantly less efficient. Optimiser
assumptions, I suppose.

Rather than duplicating most of pqGetCopyData3, I'd suggest revising
> it to take a callback, where the callback is either user-supplied
> or is supplied by PQgetCopyData to emulate the existing behavior.
> This would both avoid duplicate coding and provide a simple check that
> you've made a usable callback API (in particular, one that can do
> something sane for error cases).
>

Can do that, sure. I'll also try benchmarking a variant that doesn't take
a callback at all, but gives you the buffer pointer in addition to the
size/status return. I don't generally like callbacks.

Jeroen