Re: Speed up COPY FROM text/CSV parsing using SIMD

From: Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>
To: Nathan Bossart <nathandbossart(at)gmail(dot)com>
Cc: Greg Burd <greg(at)burd(dot)me>, Manni Wood <manni(dot)wood(at)enterprisedb(dot)com>, KAZAR Ayoub <ma_kazar(at)esi(dot)dz>, Neil Conway <neil(dot)conway(at)gmail(dot)com>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Shinya Kato <shinya11(dot)kato(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Speed up COPY FROM text/CSV parsing using SIMD
Date: 2026-08-31 17:48:05
Message-ID: CAN55FZ1utgaKb1DbW7tr+2=s4Hm1yiCEeNDY9m=qP_M_drafCg@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On Mon, 31 Aug 2026 at 17:24, Nathan Bossart <nathandbossart(at)gmail(dot)com> wrote:
>
> Claude found this bug:
>
> CREATE TABLE t (a TEXT);
> COPY t FROM PROGRAM $$printf 'abcdefghijklmnopqrst\n\\.\nxxxxxxxxxxxx\200'$$;
>
> fails with
>
> ERROR: invalid byte sequence for encoding "UTF8": 0x80
> CONTEXT: COPY t, line 2
>
> even though COPY should ignore everything after the \.

Nice catch! I can reproduce the problem and confirm that the patch fixes it.

> The best fix I could find involves teaching CopyLoadInputBuf() to avoid
> erroring in the read-ahead path. AFAICT that doesn't meaningfully change
> the performance characteristics. Patch attached.

That seems like the easiest fix without complicating the code. I will
review this code in more detail tomorrow, but I think adding the
reproducer above as a test case makes sense to me.

--
Regards,
Nazir Bilal Yavuz
Microsoft

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Matheus Alcantara 2026-08-31 18:07:07 Re: Enable partitionwise join for partition keys wrapped by RelabelType
Previous Message Paul A Jungwirth 2026-08-31 17:44:12 Re: FOR PORTION OF assertion failure in ExecInitPartitionInfo()