Re: [PATCH] Simple progress reporting for COPY command

From: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
To: Josef Šimánek <josef(dot)simanek(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [PATCH] Simple progress reporting for COPY command
Date: 2021-01-04 23:45:57
Message-ID: 6e7b056e-faff-40f6-730d-3ad6d2d1b5e9@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I did take a quick look today, and I have a couple minor comments:

1) The catalog sgml docs seem to mention bytes_processed twice (one of
that should be bytes_total), and line_processed (should be "lines_").

2) I'm not quite sure about not including any info about the command.
For example pg_stat_progress_create_index includes info about the
command, although it's not very detailed. Not sure how useful would it
be to show just COPY FROM / COPY TO, without more details.

It's probably possible to extract this from pg_stat_activity, but that
may be rather cumbersome (parsing arbitrary SQL and all that). Also,
what if the COPY is called from a function etc.?

3) I wonder if we should have something like lines_estimated. For COPY
TO it's pretty simple to calculate it as

bytes_total / (bytes_processed / lines_processed)

but what about

COPY (query) TO file

In that case we don't know the total amount of bytes / rows, so we can't
calculate any estimate. So maybe we could peek into the query plan? But
I agree this is something we can add later.

4) This comment is a bit confusing, as it mixes "total" and "processed".
I'd just say "number of bytes processed so far" instead.

Other than that, it seems fine. I'm sure we could add more features, but
it seems like a good start - I plan to get this committed once I get a
patch fixing the docs issues.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Smith 2021-01-05 00:06:24 Fix typo in comment
Previous Message Thomas Munro 2021-01-04 23:42:10 Re: language cleanups in code and docs