Re: Allow progress tracking of sub-commands

From: Manu <manuelreyesbravo(at)gmail(dot)com>
To: Antonin Houska <ah(at)cybertec(dot)at>
Cc: Alberto Piai <alberto(dot)piai(at)gmail(dot)com>, Rahila Syed <rahilasyed90(at)gmail(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, Álvaro Herrera <alvherre(at)kurilemu(dot)de>, pgsql-hackers(at)lists(dot)postgresql(dot)org, bharath(dot)rupireddyforpostgres(at)gmail(dot)com, mihailnikalayeu(at)gmail(dot)com, adam8157(at)gmail(dot)com
Subject: Re: Allow progress tracking of sub-commands
Date: 2026-10-30 03:37:20
Message-ID: 179333144055.1482692.13934879979000482281@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Antonin, Alberto,

On Wed, Jul 22, 2026 at 11:19 PM Alberto Piai
<alberto(dot)piai(at)gmail(dot)com> wrote:
> Sorry, I could have been clearer. I was referring specifically to the
> Assert(), which would be hit in the edge case you mentioned (a command
> trying to report progress past the 2nd level of nesting).

v01 fixes a case not mentioned in this thread: ANALYZE of a file_fdw
table never shows in pg_stat_progress_analyze, on 14, 18 and master.
It makes other cases worse, and that edge case is reachable from a
plain trigger. I sampled both views every ~0.5 ms, on v01 and on its
base 994f770a0fd (same as master); script and results are attached.

Fixed: ANALYZE of a file_fdw table, of a partitioned table with a
file_fdw partition, a database-wide ANALYZE, and COPY FROM whose
trigger reads a file_fdw table.

Worse:
- COPY (SELECT a FROM ft) TO shows "COPY FROM FILE relid=ft", 0 bytes.
The scan starts in ExecutorStart (copyto.c:1039), before COPY TO
(copyto.c:1227); both levels are COPY, the view takes level 1
(pgstatfuncs.c:326), and all updates go to level 2.
- Two file_fdw scans in one statement show no row, and so does one
scan after an EXPLAIN of another file_fdw query: file_fdw.c:721
turns tracking off when fdw_nodes > 1, and fdw_nodes is counted at
plan time (file_fdw.c:634).

Three levels, on an assert-enabled build:

CREATE EXTENSION file_fdw;
CREATE SERVER s FOREIGN DATA WRAPPER file_fdw;
CREATE FOREIGN TABLE ft (a int) SERVER s OPTIONS (program 'seq 3');
CREATE TABLE t (a int);
CREATE FUNCTION f() RETURNS trigger LANGUAGE plpgsql AS
$$BEGIN COPY (SELECT a FROM ft) TO '/dev/null'; RETURN NEW; END$$;
CREATE TRIGGER tr BEFORE INSERT ON t FOR EACH ROW EXECUTE FUNCTION f();
COPY t FROM PROGRAM 'echo 1';

TRAP: failed Assert("false"), File: "backend_progress.c", Line: 62,
from BeginCopyTo() in the trigger. Without asserts nothing crashes,
but each end then clears one level too high.

Fujii, Álvaro: for 14-19, would a fix at the call site like
4b445479f9e be acceptable while this goes on for master, i.e. file_fdw
not reporting COPY progress when another command is already reported?
I can write it.

Manu

Attachment Content-Type Size
cf6958-matrix.sh.txt text/plain 7.2 KB
cf6958-matrix-results.txt text/plain 12.5 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Manu 2026-10-30 04:03:47 Re: O(N^3) planning time in choose_plan_name() since 8c49a484e8e
Previous Message shihao zhong 2026-10-30 03:09:43 Data corruption: checkpoint can lose a multixact created during its start