Re: [HACKERS] PATCH: Batch/pipelining support for libpq

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
Cc: Justin Pryzby <pryzby(at)telsasoft(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Craig Ringer <craig(dot)ringer(at)enterprisedb(dot)com>, Matthieu Garrigues <matthieu(dot)garrigues(at)gmail(dot)com>, Zhihong Yu <zyu(at)yugabyte(dot)com>, Ibrar Ahmed <ibrar(dot)ahmad(at)gmail(dot)com>
Subject: Re: [HACKERS] PATCH: Batch/pipelining support for libpq
Date: 2021-03-11 18:49:02
Message-ID: 2311038.1615488542@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Alvaro Herrera <alvherre(at)2ndquadrant(dot)com> writes:
> [ v35-libpq-pipeline.patch ]

I think the changes in pqParseInput3() are broken. You should have
kept the else-structure as-is and inserted the check for "not really
idle" inside the else-clause that reports an error. As it stands,
after successfully processing an asynchronously-received error or
ParameterStatus message, the added code will cause us to return without
advancing inStart, creating an infinite loop of reprocessing that message.

It's possible that we should redefine the way things happen so that if
we're waiting for another pipeline event, we should hold off processing
of async error & ParameterStatus; but in that case you should have added
the pre-emptive return ahead of that if/else structure, where the existing
"If not IDLE state, just wait ..." test is. My guess though is that we
do need to process error messages in that state, so that the correct
patch looks more like

else
{
+ /*
+ * We're notionally not-IDLE when in pipeline mode we have
+ * completed processing the results of one query and are waiting
+ * for the next one in the pipeline. In this case, as above, just
+ * wait.
+ */
+ if (conn->asyncStatus == PGASYNC_IDLE &&
+ conn->pipelineStatus != PQ_PIPELINE_OFF &&
+ conn->cmd_queue_head != NULL)
+ return;
+
pqInternalNotice(&conn->noticeHooks,
"message type 0x%02x arrived from server while idle",
id);
/* Discard the unexpected message */
conn->inCursor += msgLength;
}

It'd be appropriate to do more than nothing to the comment block above
this if/else chain, too, because really that one ought to explain why we
should consume ERROR when in pipeline state.

(I've not looked at the rest of this patch, just scanned what you did
in fe-protocol3.c, because I wondered if there would be any interaction
with the where-to-advance-inStart changes I'm about to commit. Looks
okay modulo this issue.)

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2021-03-11 18:58:47 Re: A qsort template
Previous Message Justin Pryzby 2021-03-11 18:25:26 Re: [HACKERS] Custom compression methods