| From: | Ayoub Kazar <ayoub(dot)kazar(at)data-bene(dot)io> |
|---|---|
| To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Subject: | file_fdw: Fix ANALYZE does not enforce REJECT_LIMIT with on_error 'ignore' |
| Date: | 2026-11-05 00:40:05 |
| Message-ID: | 6cc479df-7571-4393-a572-f46dba1bfcbf@data-bene.io |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
When a file_fdw table is configured with on_error 'ignore' and a
reject_limit option,
SELECT queries properly abort with an ERROR once the number of bad rows
exceeds the limit.
However, ANALYZE silently ignores reject_limit, skips all
malformed rows, and finishes with a NOTICE.
Reproduce with this:
printf 'x\ny\n' > /tmp/fdw.csv
CREATE EXTENSION file_fdw;
CREATE SERVER fs FOREIGN DATA WRAPPER file_fdw;
CREATE FOREIGN TABLE ft_bad (a int) SERVER fs
OPTIONS (filename '/tmp/fdw.csv', format 'csv',
on_error 'ignore', reject_limit '1');
SELECT * FROM ft_bad;
ERROR: skipped more than REJECT_LIMIT (1) rows due to data type
incompatibility
CONTEXT: COPY ft_bad, line 2, column a: "y"
ANALYZE ft_bad;
NOTICE: 2 rows were skipped due to data type incompatibility
The fix is straightforward, we just add the same
cstate->opts.reject_limit check to
file_acquire_sample_rows(), just like we do for fileIterateForeignScan().
I thought about whether this is necessary at all or not (like it's not a
crash or anything, just not intended behavior at the very least), i'm
more into the first, thoughts ?
Regards,
Ayoub Kazar
| Attachment | Content-Type | Size |
|---|---|---|
| 0001-file_fdw-Enforce-REJECT_LIMIT-during-ANALYZE.patch | text/x-patch | 2.8 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Sami Imseih | 2026-11-05 01:08:54 | Re: pgstat: Flush some statistics within running transactions, take 2 |
| Previous Message | Michael Paquier | 2026-11-05 00:15:01 | Re: [PATCH] pgstat: Add helper to drop entries from a stats kind's hash |