BUG #15975: Copy from large text files on Windows returns "unknown error"

From: PG Bug reporting form <noreply(at)postgresql(dot)org>
To: pgsql-bugs(at)lists(dot)postgresql(dot)org
Cc: johann(at)myrkraverk(dot)com
Subject: BUG #15975: Copy from large text files on Windows returns "unknown error"
Date: 2019-08-23 21:11:38
Message-ID: 15975-80e6438a9a73f74f@postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

The following bug has been logged on the website:

Bug reference: 15975
Logged by: Johann Oskarsson
Email address: johann(at)myrkraverk(dot)com
PostgreSQL version: 11.5
Operating system: Windows 10
Description:

When I try to load a large text file into PostgreSQL 11 on Windows, I get an
"unknown error." This has been discussed on StackOverflow here:
https://stackoverflow.com/q/53523051/264751

It turns out that trying to COPY FROM a large textfile, say several
gigabytes, will always fail. According to Stack Overflow, this only happens
on PostgreSQL 11. The earlier releases don't seem to have this problem.

Example, given

create table pwned( passwd sha1 not null, count int4 not null );

Then

copy pwned( passwd, count ) from
'c:\src\postgres\pwned\pwned-passwords-sha1-ordered-by-hash-v5.txt' with (
format text, delimiter ':' );

gives

ERROR: could not stat file
"c:\src\postgres\pwned\pwned-passwords-sha1-ordered-by-hash-v5.txt": Unknown
error

Workaround is to pipe the file through a program, as discussed here:
https://wp.me/p4w9PF-di

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Tom Lane 2019-08-23 21:49:20 Re: BUG #15858: could not stat file - over 4GB
Previous Message David G. Johnston 2019-08-23 18:09:56 Re: BUG #15974: Concact with || doesn't work, but function CONCAT () works