bug: copy progress reporting of backends which run multiple COPYs

From: Justin Pryzby <pryzby(at)telsasoft(dot)com>
To: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Cc: Josef Šimánek <josef(dot)simanek(at)gmail(dot)com>, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: bug: copy progress reporting of backends which run multiple COPYs
Date: 2023-01-19 05:47:03
Message-ID: 20230119054703.GB13860@telsasoft.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

pg_stat_progress_copy was added in v14 (8a4f618e7, 9d2d45700).

But if a command JOINs file_fdw tables, the progress report gets bungled
up. This will warn/assert during file_fdw tests.

diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6743e68cef6..7abcb4f60db 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -10,6 +10,7 @@
*/
#include "postgres.h"

+#include "commands/progress.h"
#include "port/atomics.h" /* for memory barriers */
#include "utils/backend_progress.h"
#include "utils/backend_status.h"
@@ -105,6 +106,20 @@ pgstat_progress_end_command(void)
if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
return;

+// This currently fails file_fdw tests, since pgstat_progress evidently fails
+// to support simultaneous copy commands, as happens during JOIN.
+ /* bytes progress is not available in all cases */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_COPY &&
+ beentry->st_progress_param[PROGRESS_COPY_BYTES_TOTAL] > 0)
+ {
+ volatile int64 *a = beentry->st_progress_param;
+ if (a[PROGRESS_COPY_BYTES_PROCESSED] > a[PROGRESS_COPY_BYTES_TOTAL])
+ elog(WARNING, "PROGRESS_COPY_BYTES_PROCESSED %ld %ld",
+ a[PROGRESS_COPY_BYTES_PROCESSED],
+ a[PROGRESS_COPY_BYTES_TOTAL]);
+ // Assert(a[PROGRESS_COPY_BYTES_PROCESSED] <= a[PROGRESS_COPY_BYTES_TOTAL]);
+ }
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
beentry->st_progress_command_target = InvalidOid;

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2023-01-19 06:01:35 Re: [PATCH] random_normal function
Previous Message Andrey Borodin 2023-01-19 05:45:15 Re: Experiments with Postgres and SSL