heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

From: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)
Date: 2018-09-19 00:21:17
Message-ID: 15d90682-0c22-e7d4-7667-7ceaef075454@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I've been doing some testing today, and it seems heap_sync is somewhat
confused by partitioned tables. I'm doing a COPY into a partitioned
table (lineitem from TPC-H partitioned per month) like this:

---------------------------------------------------------------------
BEGIN;

CREATE TABLE lineitem (...) PARTITION BY RANGE (l_shipdate);

CREATE TABLE lineitem_1992_01 PARTITION OF lineitem FOR VALUES FROM
('1992-01-01') TO ('1992-02-01');

...

CREATE TABLE lineitem_1998_12 PARTITION OF lineitem FOR VALUES FROM
('1998-12-01') TO ('1999-01-01');

COPY INTO lineitem FROM ....

COMMIT;
---------------------------------------------------------------------

I'm observing COPY failing like this:

ERROR: could not open file "base/16384/16427": No such file or directory

where the relfilenode matches the "lineitem" relation (which is
guaranteed to be empty, as it's partitioned, so it's not surprising the
relfilenode does not exist).

After adding an Assert(false) into mdopen(), which is where the error
comes from, I got this backtrace (full backtrace attached):

...
#3 0x0000000000766387 in mdopen (...) at md.c:609
#4 0x00000000007674b7 in mdnblocks (...) at md.c:876
#5 0x0000000000767a61 in mdimmedsync (...) at md.c:1034
#6 0x00000000004c62b5 in heap_sync (...) at heapam.c:9399
#7 0x00000000005a2bdd in CopyFrom (...) at copy.c:2890
#8 0x00000000005a1621 in DoCopy (...) at copy.c:992
...

So apparently CopyFrom() invokes heap_sync() on the partitioned
relation, which attempts to do mdimmedsync() only on the root. That
seems like a bug to me.

Obviously this only applies to wal_level=minimal. There are multiple
callers of heap_sync(), but only the COPY seems to be affected by this,
because the rest can't see partitioned tables.

So it seems heap_sync() needs to be improved to sync all partitions.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment Content-Type Size
backtrace.txt text/plain 10.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Rowley 2018-09-19 00:59:38 incorrect comment or possible lock upgrade hazards in executor
Previous Message Tsunakawa, Takayuki 2018-09-19 00:14:57 RE: Changing the setting of wal_sender_timeout per standby