pgsql: Optimize COPY FROM (FORMAT {text,csv}) using SIMD.

From: Nathan Bossart <nathan(at)postgresql(dot)org>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Optimize COPY FROM (FORMAT {text,csv}) using SIMD.
Date: 2026-03-13 16:08:02
Message-ID: E1w153S-003r6u-18@gemulon.postgresql.org
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Optimize COPY FROM (FORMAT {text,csv}) using SIMD.

Presently, such commands scan the input buffer one byte at a time
looking for special characters. This commit adds a new path that
uses SIMD instructions to skip over chunks of data without any
special characters. This can be much faster.

To avoid regressions, SIMD processing is disabled for the remainder
of the COPY FROM command as soon as we encounter a short line or a
special character (except for end-of-line characters, else we'd
always disable it after the first line). This is perhaps too
conservative, but it could probably be made more lenient in the
future via fine-tuned heuristics.

Author: Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>
Co-authored-by: Shinya Kato <shinya11(dot)kato(at)gmail(dot)com>
Reviewed-by: Ayoub Kazar <ma_kazar(at)esi(dot)dz>
Reviewed-by: Andrew Dunstan <andrew(at)dunslane(dot)net>
Reviewed-by: Neil Conway <neil(dot)conway(at)gmail(dot)com>
Reviewed-by: Greg Burd <greg(at)burd(dot)me>
Tested-by: Manni Wood <manni(dot)wood(at)enterprisedb(dot)com>
Tested-by: Mark Wong <markwkm(at)gmail(dot)com>
Discussion: https://postgr.es/m/CAOzEurSW8cNr6TPKsjrstnPfhf4QyQqB4tnPXGGe8N4e_v7Jig%40mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/e0a3a3fd5361913502ff696ecf47770ca55975ae

Modified Files
--------------
src/backend/commands/copyfrom.c | 1 +
src/backend/commands/copyfromparse.c | 185 ++++++++++++++++++++++++++++++-
src/include/commands/copyfrom_internal.h | 1 +
3 files changed, 184 insertions(+), 3 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Nathan Bossart 2026-03-13 16:32:44 pgsql: Initialize variable to placate compiler.
Previous Message Peter Eisentraut 2026-03-13 15:23:19 pgsql: Factor out constructSetOpTargetlist() from transformSetOperation