| From: | Nathan Bossart <nathandbossart(at)gmail(dot)com> |
|---|---|
| To: | Greg Burd <greg(at)burd(dot)me> |
| Cc: | Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>, Manni Wood <manni(dot)wood(at)enterprisedb(dot)com>, KAZAR Ayoub <ma_kazar(at)esi(dot)dz>, Neil Conway <neil(dot)conway(at)gmail(dot)com>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Shinya Kato <shinya11(dot)kato(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: Speed up COPY FROM text/CSV parsing using SIMD |
| Date: | 2026-08-31 14:24:47 |
| Message-ID: | apWOr1kgVSqweBnU@nathan |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Claude found this bug:
CREATE TABLE t (a TEXT);
COPY t FROM PROGRAM $$printf 'abcdefghijklmnopqrst\n\\.\nxxxxxxxxxxxx\200'$$;
fails with
ERROR: invalid byte sequence for encoding "UTF8": 0x80
CONTEXT: COPY t, line 2
even though COPY should ignore everything after the \.
The best fix I could find involves teaching CopyLoadInputBuf() to avoid
erroring in the read-ahead path. AFAICT that doesn't meaningfully change
the performance characteristics. Patch attached.
--
nathan
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-Make-COPY-FROM-s-SIMD-read-ahead-speculative.patch | text/plain | 4.5 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Nathan Bossart | 2026-08-31 14:34:17 | Re: WAIT FOR command should do some query jumbling |
| Previous Message | Matthias van de Meent | 2026-08-31 14:23:05 | Re: Commit Sequence Numbers and Visibility |