Re: Progress reporting for pg_verify_checksums

From: Michael Banck <michael(dot)banck(at)credativ(dot)de>
To: Fabien COELHO <fabien(dot)coelho(at)mines-paristech(dot)fr>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Bernd Helmle <mailings(at)oopsware(dot)de>, Michael Paquier <michael(at)paquier(dot)xyz>, Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Progress reporting for pg_verify_checksums
Date: 2019-03-12 14:13:33
Message-ID: 1552400013.4947.56.camel@credativ.de
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

Hi,

Am Dienstag, den 19.02.2019, 16:37 +0100 schrieb Fabien COELHO
>
> About :
>
> total_percent = total_size ? (int64) ((current_size / MEGABYTES) * 100 / (total_size / MEGABYTES)) : 0;
>
> MEGABYTES can be simplified and will enhance precision. ISTM that the
> percent could be a double:
>
> total_percent = total_size ? 100.0 * current_size / total_size : 0.0 ;
>
> and then just let fprintf do the rounding.

Ok, done so.

> I would bother rounding down < 100% to 100, because then you would get
>
> 1560/1492 MB (100\%, X MB/s)
>
> which is kind of silly.

No, we cap the total_size to current_size so you won't see that (but
total_size will potentially gradually increase). pg_basebackup has the
same behaviour.

> I'd still would use more precise than one second precision time so that
> the speed displayed at the beginning is more realistic. eg I got on a
> short test with probably in system cache files:
>
> 188/188 MB (100%, 188 MB/s)
>
> but the reality is that it took 0.126 seconds, and the speed was really
> 1492 MB/s.

Because I implemented I/O throttling for pg_checksums and needed it
there anyway, I've switched it to the instr_time API now. It now updates
the progress 10 times every second.

So now I get

|20/20 MB (100%, 1382 MB/s)

for an empty cluster vs

|20/20 MB (100%, 20 MB/s)

with the previous version of the patch.

New patch attached.

Michael

--
Michael Banck
Projektleiter / Senior Berater
Tel.: +49 2166 9901-171
Fax: +49 2166 9901-100
Email: michael(dot)banck(at)credativ(dot)de

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer

Unser Umgang mit personenbezogenen Daten unterliegt
folgenden Bestimmungen: https://www.credativ.de/datenschutz

Attachment Content-Type Size
pg_verify_checksums_progress_V10.patch text/x-patch 8.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2019-03-12 14:29:17 Re: pgsql: Removed unused variable, openLogOff.
Previous Message Evgeniy Efimkin 2019-03-12 13:48:27 Re: Special role for subscriptions