Hey Nazir, Nathan,
Thanks for turning this around so fast. The v1 looks right to me.
Checking only for the backslash and letting the scalar parser do the
actual marker recognition is the right call. Trying to detect a full
'\.' in the SIMD helper would mean duplicating the CRLF handling and the
"marker is not alone on its line" checks, and that logic is fiddly enough
that I'd rather it live in exactly one place. The !is_csv guard is also
correctly scoped: the scalar path only treats '\' / '\.' specially in
text mode (copyfromparse.c, "if (c == '\\' && !is_csv)"), so CSV never
had a buffered marker to worry about and doesn't need the check.
On the perf side, I read your 1-2% on TEXT-wide as the expected cost:
the tail check bails to scalar on any backslash in the last < sizeof(Vector8)
bytes, not just on a real marker, so escaped data near a buffer boundary
gives up SIMD for that refill. It re-enables on the next line, so it's
confined to boundaries, and as you say it's still well ahead of pre-SIMD.
I think that's the right trade; correctness wins here. Might be worth a
sentence in the commit message noting the small text-path cost so it's on
the record.
One thing I'd like to see land with the fix: a test. I'll see if I can
knock one out if you don't first.
best,
-greg