BUG #15333: pg_dump error on large table -- "pg_dump: could not stat file...Unknown error"

From: PG Bug reporting form <noreply(at)postgresql(dot)org>
To: pgsql-bugs(at)lists(dot)postgresql(dot)org
Cc: mark(dot)lai(at)integrafec(dot)com
Subject: BUG #15333: pg_dump error on large table -- "pg_dump: could not stat file...Unknown error"
Date: 2018-08-16 12:51:54
Message-ID: 153442391458.1505.9181095584291689853@wrigleys.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

The following bug has been logged on the website:

Bug reference: 15333
Logged by: Mark Lai
Email address: mark(dot)lai(at)integrafec(dot)com
PostgreSQL version: 10.4
Operating system: Windows Server 2016 Server
Description:

Hi,

When I pg_dump a large table (> 35 GB), I get the following error message:

pg_dump: could not stat file
"O:\postgres-server3\test#test#large_test/2793.dat.gz": Unknown error

The dump however appears to restore correctly.

**************************************
**Script used to create the table**
**************************************
CREATE SCHEMA IF NOT EXISTS test;

DROP TABLE IF EXISTS test.large_test;
CREATE UNLOGGED TABLE test.large_test (num1 bigint, num2 double precision,
num3 double precision);

INSERT INTO test.large_test (num1, num2, num3)
SELECT round(random()*10), random(), random()*142
FROM generate_series(1, 40*20000000) s(i);
--20000000 ~ 1GB

*****************************************************
**power shell script used to launch pg_dump**
*****************************************************
$database = "test"
$schema = "test"
$table = "large_test"

$baseOutputDirectory = "O:\postgres-server3\"
$baseLogDirectory = "O:\postgres-server3\logs\"

############################

New-Item -ItemType Directory -Force -Path $baseOutputDirectory
New-Item -ItemType Directory -Force -Path $baseLogDirectory

$schemaTable = "$($database)#$($schema)#$($table)"
$outputDirectory = "$($baseOutputDirectory)$($schemaTable)"
$logFile = "$($baseLogDirectory)$($schemaTable).log"

Remove-Item $outputDirectory -Force -Recurse -ErrorAction Ignore
Remove-Item $logFile -Force -Recurse -ErrorAction Ignore

Start-Job -Name $schemaTable -ScriptBlock{& "C:\Program
Files\PostgreSQL\10\bin\pg_dump.exe" --verbose --host=localhost --port=5432
--username=test123123 --no-password --jobs=1 --format=directory
--table=$Using:schema.$Using:table --file=$Using:outputDirectory
$Using:database 1> $Using:logFile 2>&1 }

************
**Log file**
*************
pg_dump.exe : pg_dump: last built-in OID is 16383
At line:1 char:1
+ & "C:\Program Files\PostgreSQL\10\bin\pg_dump.exe" --verbose --host=l
...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (pg_dump: last built-in OID is
16383:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError

pg_dump: reading extensions
pg_dump: identifying extension members
pg_dump: reading schemas
pg_dump: reading user-defined tables
pg_dump: reading user-defined functions
pg_dump: reading user-defined types
pg_dump: reading procedural languages
pg_dump: reading user-defined aggregate functions
pg_dump: reading user-defined operators
pg_dump: reading user-defined access methods
pg_dump: reading user-defined operator classes
pg_dump: reading user-defined operator families
pg_dump: reading user-defined text search parsers
pg_dump: reading user-defined text search templates
pg_dump: reading user-defined text search dictionaries
pg_dump: reading user-defined text search configurations
pg_dump: reading user-defined foreign-data wrappers
pg_dump: reading user-defined foreign servers
pg_dump: reading default privileges
pg_dump: reading user-defined collations
pg_dump: reading user-defined conversions
pg_dump: reading type casts
pg_dump: reading transforms
pg_dump: reading table inheritance information
pg_dump: reading event triggers
pg_dump: finding extension tables
pg_dump: finding inheritance relationships
pg_dump: reading column info for interesting tables
pg_dump: finding the columns and types of table "test.large_test"
pg_dump: flagging inherited columns in subtables
pg_dump: reading indexes
pg_dump: reading extended statistics
pg_dump: reading constraints
pg_dump: reading triggers
pg_dump: reading rewrite rules
pg_dump: reading policies
pg_dump: reading row security enabled for table "test.large_test"
pg_dump: reading policies for table "test.large_test"
pg_dump: reading publications
pg_dump: reading publication membership
pg_dump: reading publication membership for table "test.large_test"
pg_dump: reading subscriptions
pg_dump: reading dependency data
pg_dump: saving encoding = UTF8
pg_dump: saving standard_conforming_strings = on
pg_dump: saving search_path =
pg_dump: dumping contents of table "test.large_test"
pg_dump: could not stat file
"O:\postgres-server3\test#test#large_test/2793.dat.gz": Unknown error

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message PG Bug reporting form 2018-08-16 14:05:17 BUG #15334: Partition elimination not working as expected when using enum as partition key
Previous Message Bruce Momjian 2018-08-16 12:33:12 Re: [PG_UPGRADE] 9.6 to 10.5