Re: libpq: PQgetCopyData() and allocation overhead

On Fri, 3 Mar 2023 at 18:14, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Jeroen Vermeulen <jtvjtv(at)gmail(dot)com> writes:
> > On Fri, 3 Mar 2023 at 17:33, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> Let's not do that. Declare it const char *, or maybe better const void
> *.
>
> > Personally I would much prefer "char" over "void" here:
> > * It really is a char buffer, containing text.
>
> Not in binary-mode COPY.
>

True. And in that case zero-termination doesn't matter much either. But
overall libpq's existing choice seems reasonable.

> As for const, I would definitely have preferred that. But if the caller
> > needs a zero-terminated string, forcing them into a memcpy() would kind
> of
> > defeat the purpose.
>
> I'm willing to grant that avoiding malloc-and-free is worth the trouble.
> I'm not willing to allow applications to scribble on libpq buffers to
> avoid memcpy. Even your not-a-patch patch fails to make the case that
> this is essential, because you could have used fwrite() instead of
> printf() (which would be significantly faster yet btw, printf formatting
> ain't cheap).
>

Your house, your rules. For my own use-case "const" is just peachy.

The printf() is just the simplest example that sprang to mind though.
There may be other use-cases out there involving libraries that require
zero-terminated strings, and I figured an ability to set a sentinel could
help those.

> Can do that, sure. I'll also try benchmarking a variant that doesn't take
> > a callback at all, but gives you the buffer pointer in addition to the
> > size/status return. I don't generally like callbacks.
>
> Um ... that would require an assumption that libpq neither changes nor
> moves that buffer before returning to the caller. I don't much like
> that either.
>

Not an assumption about _before returning to the caller_ I guess, because
the function would be on top of that anyway. The concern would be libpq
changing or moving the buffer _before the caller is done with the line._
Which would require some kind of clear rule about what invalidates the
buffer. Yes, that is easier with the callback.

Jeroen