Re: Libpq COPY optimization

I'll send it to -patches shortly

Alon.

On 1/23/06 10:20 PM, "Bruce Momjian" <pgman(at)candle(dot)pha(dot)pa(dot)us> wrote:

>
> Can I get an updated patch for this?
>
> ---------------------------------------------------------------------------
>
> Tom Lane wrote:
>> "Alon Goldshuv" <agoldshuv(at)greenplum(dot)com> writes:
>>> Please help me understand this better. It appears to me that when the
>>> client->backend pipe fills up, pqSendSome() consumes any incoming
>>> NOTICE/WARNING messages before waiting, which should prevent deadlock.
>>
>> Hm, I had forgotten that the low-level pqSendSome routine does that.
>> That makes the PQconsumeInput call in PQputCopyData redundant (or
>> almost; see below). The parseInput call is still needed, because it's
>> there to pull NOTICE messages out of the input buffer and get rid of
>> them, rather than possibly having the input buffer grow to exceed
>> memory. But when there's nothing for it to do, parseInput is cheap
>> enough that there's no real need to bypass it.
>>
>> In short, if you just remove the PQconsumeInput call I think you'll find
>> that it does what you want.
>>
>> The only case where it's helpful to have it there is if there's a
>> incomplete message in the input buffer, as parseInput isn't quite so
>> fast if it has to determine that the message is incomplete. Without
>> the PQconsumeInput call, the incomplete-message state could persist
>> for a long time, and you'd pay the parseInput overhead each time through
>> PQputCopyData. However, that's certainly not the normal situation,
>> so I think we could leave that case slightly pessimal. It's certainly
>> true that that path in parseInput is a lot faster than a kernel call,
>> so it'd still be better than it is now.
>>
>> regards, tom lane
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 9: In versions below 8.0, the planner will ignore your desire to
>> choose an index scan if your joining column's datatypes do not
>> match
>>