Re: [PATCH] Better Performance for PostgreSQL with large INSERTs

Hi Andres,

> Have you tried to verify that this doesn't cause performance
> regressions in
> other workloads? pq_recvbuf() has this code:
>
...
>
> I do seem to recall that just increasing the buffer size substantially
> lead to
> more time being spent inside that memmove() (likely due to exceeding
> L1/L2).

Do you have any pointers to discussions or other data about that?

My (quick) analysis was that clients that send one request,
wait for an answer, then send the next request wouldn't run that code
as there's nothing behind the individual requests that could be moved.

But yes, Pipeline Mode[1] might/would be affected.

The interesting question is how much data can userspace copy before
that means more load than doing a userspace-kernel-userspace round trip.
(I guess that moving 64kB or 128kB should be quicker, especially since
the various CPU mitigations.)

As long as there are complete requests in the buffer the memmove()
could be avoided; only the initial part of the first incomplete request
might need moving to the beginning.

That patch would be more than +- 1 line again ;)

The documentation says

> Pipelining is less useful, and more complex,
> when a single pipeline contains multiple transactions
> (see Section 32.5.1.3).

are there any benchmarks/usage statistics for pipeline mode?

Regards,

Phil

Ad 1: https://www.postgresql.org/docs/current/libpq-pipeline-mode.html