Re: BUG #15333: pg_dump error on large table -- "pg_dump: could not stat file...Unknown error"

Hi Tom,

I ran the dump on the large table with no jobs flag and got the same error.

"C:\Program Files\PostgreSQL\10\bin\pg_dump.exe" --verbose --host=localhost
--port=5432 --username=mark1492 --no-password --format=directory
--table=$Using:schema.$Using:table --file=$Using:outputDirectory
$Using:database 1> $Using:logFile 2>&1 "

****************************
*Log for large table dump
*****************************
pg_dump.exe : pg_dump: last built-in OID is 16383
At line:1 char:1
+ & "C:\Program Files\PostgreSQL\10\bin\pg_dump.exe" --verbose --host=l ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (pg_dump: last built-in OID is
16383:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError

pg_dump: reading extensions
pg_dump: identifying extension members
pg_dump: reading schemas
pg_dump: reading user-defined tables
pg_dump: reading user-defined functions
pg_dump: reading user-defined types
pg_dump: reading procedural languages
pg_dump: reading user-defined aggregate functions
pg_dump: reading user-defined operators
pg_dump: reading user-defined access methods
pg_dump: reading user-defined operator classes
pg_dump: reading user-defined operator families
pg_dump: reading user-defined text search parsers
pg_dump: reading user-defined text search templates
pg_dump: reading user-defined text search dictionaries
pg_dump: reading user-defined text search configurations
pg_dump: reading user-defined foreign-data wrappers
pg_dump: reading user-defined foreign servers
pg_dump: reading default privileges
pg_dump: reading user-defined collations
pg_dump: reading user-defined conversions
pg_dump: reading type casts
pg_dump: reading transforms
pg_dump: reading table inheritance information
pg_dump: reading event triggers
pg_dump: finding extension tables
pg_dump: finding inheritance relationships
pg_dump: reading column info for interesting tables
pg_dump: finding the columns and types of table "test.large_test"
pg_dump: flagging inherited columns in subtables
pg_dump: reading indexes
pg_dump: reading extended statistics
pg_dump: reading constraints
pg_dump: reading triggers
pg_dump: reading rewrite rules
pg_dump: reading policies
pg_dump: reading row security enabled for table "test.large_test"
pg_dump: reading policies for table "test.large_test"
pg_dump: reading publications
pg_dump: reading publication membership
pg_dump: reading publication membership for table "test.large_test"
pg_dump: reading subscriptions
pg_dump: reading dependency data
pg_dump: saving encoding = UTF8
pg_dump: saving standard_conforming_strings = on
pg_dump: saving search_path =
pg_dump: dumping contents of table "test.large_test"
pg_dump: could not stat file
"O:\postgres-server3\test#test#large_test/2796.dat.gz": Unknown error

*************************************************************************************************************

The dump was successful on a small table.

****************************
*Log for small table dump
*****************************
pg_dump.exe : pg_dump: last built-in OID is 16383
At line:1 char:1
+ & "C:\Program Files\PostgreSQL\10\bin\pg_dump.exe" --verbose --host=l ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (pg_dump: last built-in OID is
16383:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError

pg_dump: reading extensions
pg_dump: identifying extension members
pg_dump: reading schemas
pg_dump: reading user-defined tables
pg_dump: reading user-defined functions
pg_dump: reading user-defined types
pg_dump: reading procedural languages
pg_dump: reading user-defined aggregate functions
pg_dump: reading user-defined operators
pg_dump: reading user-defined access methods
pg_dump: reading user-defined operator classes
pg_dump: reading user-defined operator families
pg_dump: reading user-defined text search parsers
pg_dump: reading user-defined text search templates
pg_dump: reading user-defined text search dictionaries
pg_dump: reading user-defined text search configurations
pg_dump: reading user-defined foreign-data wrappers
pg_dump: reading user-defined foreign servers
pg_dump: reading default privileges
pg_dump: reading user-defined collations
pg_dump: reading user-defined conversions
pg_dump: reading type casts
pg_dump: reading transforms
pg_dump: reading table inheritance information
pg_dump: reading event triggers
pg_dump: finding extension tables
pg_dump: finding inheritance relationships
pg_dump: reading column info for interesting tables
pg_dump: finding the columns and types of table "test.small_test"
pg_dump: flagging inherited columns in subtables
pg_dump: reading indexes
pg_dump: reading extended statistics
pg_dump: reading constraints
pg_dump: reading triggers
pg_dump: reading rewrite rules
pg_dump: reading policies
pg_dump: reading row security enabled for table "test.small_test"
pg_dump: reading policies for table "test.small_test"
pg_dump: reading publications
pg_dump: reading publication membership
pg_dump: reading publication membership for table "test.small_test"
pg_dump: reading subscriptions
pg_dump: reading dependency data
pg_dump: saving encoding = UTF8
pg_dump: saving standard_conforming_strings = on
pg_dump: saving search_path =
pg_dump: dumping contents of table "test.small_test"

On Thu, Aug 16, 2018 at 10:25 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> =?utf-8?q?PG_Bug_reporting_form?= <noreply(at)postgresql(dot)org> writes:
> > When I pg_dump a large table (> 35 GB), I get the following error
> message:
>
> > pg_dump: could not stat file
> > "O:\postgres-server3\test#test#large_test/2793.dat.gz": Unknown error
>
> > The dump however appears to restore correctly.
>
> Hm, I assume this shows up just at the end of the dump run?
>
> The only plausible match for that error string is in the
> fsync_dir_recurse() processing that tries to force all the output files
> down to disk before reporting that the dump is complete. As long as
> you don't have an OS crash right afterwards, failure to fsync is
> harmless. Still, it's weird.
>
> Does it vary with the size of the dumped table? What if you remove
> the parallelism (no --jobs option)?
>
> regards, tom lane
>