Re: Unable to copy large (>2GB) files using PostgreSQL 11 (Windows)

From: "Daniel Verite" <daniel(at)manitou-mail(dot)org>
To: "Michael Paquier" <michael(at)paquier(dot)xyz>
Cc: "Alistair Johnson" <alistair(dot)e(dot)w(dot)j(at)gmail(dot)com>,pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: Unable to copy large (>2GB) files using PostgreSQL 11 (Windows)
Date: 2018-11-03 11:11:53
Message-ID: df939c6f-2866-48b8-b3fe-5cbb54576a53@manitou-mail.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Michael Paquier wrote:

> Windows portability problems with stat() makes this problem way harder
> than it looks at first glance, because it also uses a "stat" structure
> which is not able to store the size of files larger than 2GB

FWIW I don't reproduce the problem with a self-compiled PostgreSQL 11.0
with the MSYS2/mingw64 toolchain on a Windows 10 64-bit desktop.

But I do get the error when using PG 11.0 packaged by
EDB and trying the same test on the same machine
(that is, \copy table from file with a file a bit over 2GB).

The error seems to be emited at the point of this fstat() call in
psql/copy.c:

/* make sure the specified file is not a directory */
if ((result = fstat(fileno(copystream), &st)) < 0)
psql_error("could not stat file \"%s\": %s\n",
options->file,

Note that it's fstat() rather than stat().
A quick git grep -w seems to show that fstat() doesn't go through
the same kind of aliasing that can make stat() a macro actually
calling pgwin32_safestat().

Best regards,
--
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Tom Lane 2018-11-03 17:03:20 Re: BUG #15476: Problem on show_trgm with 4 byte UTF-8 characters
Previous Message Thomas Munro 2018-11-03 09:13:44 Re: BUG #15460: Error while creating index or constraint