Re: Speed up COPY FROM text/CSV parsing using SIMD

On Fri, Mar 6, 2026 at 12:55 PM Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>
wrote:

> Hi Manni!
>
> On Fri, 6 Mar 2026 at 21:13, Manni Wood <manni(dot)wood(at)enterprisedb(dot)com>
> wrote:
> >
> > When I ran `meson setup build --buildtype=debugoptimized` on both my x86
> machine and my arm machine, the response on both was:
> >
> > "External libraries
> > " lz4 : NO"
> >
> > However, I did not remove commit 34dfca2934 from any of my Postgres
> builds; I left that commit in place.
> >
> > Let me know if that helps!
>
> That definitely helps, thanks! If you have a chance, could you also
> run the benchmark with LZ4 enabled? I expect you may see significantly
> better performance, similar to what I observed.
>
> --
> Regards,
> Nazir Bilal Yavuz
> Microsoft
>

Hi, Nazir.

Well, golly! Look at these numbers. Old master with no lz4, your v11 patch
with no lz4, and then your v11 patch with lz4 compiled in.

(Aside: I assume everything is good for my lz4 build after installing the
lz4 dev library and seeing this with my meson config:

" External libraries
" lz4 : YES 1.9.4"

and this from the db itself:

$ ./bin/psql -U mwood -d postgres -c 'show default_toast_compression'
default_toast_compression
---------------------------
lz4
)

x86 NARROW old master
TXT : 25909.060500 ms
CSV : 28137.591250 ms
TXT with 1/3 escapes: 27794.177000 ms
CSV with 1/3 quotes: 34541.704750 ms

x86 NARROW v11
TXT : 27212.945750 ms -5.032545% regression
CSV : 26985.971250 ms 4.092817% improvement
TXT with 1/3 escapes: 27216.510000 ms 2.078374% improvement
CSV with 1/3 quotes: 32817.267500 ms 4.992334% improvement

x86 NARROW v11 lz4
TXT : 26471.776500 ms -2.171889% regression
CSV : 25607.026250 ms 8.993538% improvement
TXT with 1/3 escapes: 28628.729750 ms -3.002617% regression
CSV with 1/3 quotes: 34729.006750 ms -0.542249% regression

x86 WIDE old master
TXT : 28778.426500 ms
CSV : 35671.908000 ms
TXT with 1/3 escapes: 32441.549750 ms
CSV with 1/3 quotes: 47024.416000 ms

x86 WIDE v11
TXT : 22571.305750 ms 21.568659% improvement
CSV : 22711.524750 ms 36.332184% improvement
TXT with 1/3 escapes: 29236.453000 ms 9.879604% improvement
CSV with 1/3 quotes: 40022.110750 ms 14.890786% improvement

x86 WIDE v11 lz4
TXT : 8032.912750 ms 72.087033% improvement
CSV : 8047.098000 ms 77.441358% improvement
TXT with 1/3 escapes: 15428.139500 ms 52.443272% improvement
CSV with 1/3 quotes: 27517.084500 ms 41.483410% improvement

arm NARROW old master
TXT : 10997.568250 ms
CSV : 10797.549000 ms
TXT with 1/3 escapes: 10299.047000 ms
CSV with 1/3 quotes: 12559.385750 ms

arm NARROW v11
TXT : 10340.966250 ms 5.970429% improvement
CSV : 10224.399500 ms 5.308144% improvement
TXT with 1/3 escapes: 10438.216750 ms -1.351288% regression
CSV with 1/3 quotes: 11865.934000 ms 5.521383% improvement

arm NARROW v11 lz4
TXT : 9783.737000 ms 11.037270% improvement
CSV : 10122.890750 ms 6.248254% improvement
TXT with 1/3 escapes: 10298.780250 ms 0.002590% improvement
CSV with 1/3 quotes: 11738.992250 ms 6.532115% improvement

arm WIDE old master
TXT : 11825.771250 ms
CSV : 13907.074000 ms
TXT with 1/3 escapes: 13430.691250 ms
CSV with 1/3 quotes: 17557.954500 ms

arm WIDE v11
TXT : 9001.442250 ms 23.882831% improvement
CSV : 8940.928750 ms 35.709490% improvement
TXT with 1/3 escapes: 12049.668500 ms 10.282589% improvement
CSV with 1/3 quotes: 15277.843250 ms 12.986201% improvement

arm WIDE v11 lz4
TXT : 3186.825500 ms 73.051859% improvement
CSV : 3142.526500 ms 77.403396% improvement
TXT with 1/3 escapes: 6180.176000 ms 53.984677% improvement
CSV with 1/3 quotes: 9460.505500 ms 46.118407% improvement

Cheers,

-Manni

--
-- Manni Wood EDB: https://www.enterprisedb.com