Re: Speed up COPY FROM text/CSV parsing using SIMD

On Tue, Sep 8, 2026, at 11:36 AM, Nathan Bossart wrote:
> On Tue, Sep 08, 2026 at 11:08:36AM -0400, Greg Burd wrote:
>> One thing I'd like to see land with the fix: a test. I'll see if I can
>> knock one out if you don't first.
>
> I'm open to adding a test, but I'm not going to hold up the patch for it.
> I've been reluctant to add too many tests for corner cases here because I'm
> concerned about portability and fragility. But I will admit that I haven't
> tried too hard...

Hey Nathan, I like your approach better.

The test writes one full input buffer (INPUT_BUF_SIZE, 65536 bytes) down a
COPY FROM PROGRAM pipe: a 65524-byte line, then a complete \. marker, then
8 trailing bytes. That leaves 11 bytes unexamined when the SIMD path wants
to refill, which is under sizeof(Vector8) and already holds the whole
marker. The writer then dribbles a byte every 50ms and keeps the pipe open,
so a speculative load blocks rather than seeing EOF. statement_timeout
bounds the failure, so a regression fails the test instead of hanging the
suite. Skipped on Windows, where the blocking pipe and SIGPIPE behavior
isn't reliable.

I confirmed both directions, with the fix reverse-applied it fails with
"canceling statement due to statement timeout" at COPY line 2, and with the
fix in place it passes in under 2s. I also instrumented the early return in
CopyLoadInputBuf() to be sure the test actually reaches it, and it fires
once with nbytes=11.

Attached is a TAP test and your patch as v3.

-greg

> --
> nathan