Re: pgbench logging broken by time logic changes

From: Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr>
To: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Cc: Michael Paquier <michael(at)paquier(dot)xyz>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Gregory Smith <gregsmithpgsql(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, david(dot)christensen(at)crunchydata(dot)com
Subject: Re: pgbench logging broken by time logic changes
Date: 2021-07-11 13:07:09
Message-ID: alpine.DEB.2.22.394.2107111201560.1105232@pseudo
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Hello Thomas,

> I committed the code change without the new TAP tests, because I
> didn't want to leave the open item hanging any longer.

Ok. Good.

> As for the test, ... [...]

Argh, so there are no tests that would have caught the regressions:-(

> ... I know it can fail, and your v18 didn't fix that, because...
>
> +check_pgbench_logs($bdir, '001_pgbench_log_1', $nthreads, 1, 3,
> ... this range can be exceeded.

Indeed. I meant to move that one in the TODO section as well, not just the
previous call, so that all time-sensitive tests are fully ignored but
reported, which would be enough for me.

> I suspect the number of aggregates could be made deterministic, as I
> described in an earlier message. What do you think about doing
> something like that first for the next release, before trying to add
> assertions about the number of aggregates?

I think that last time I did something to get more deterministic results
in pgbench, which involved a few lines of hocus-pocus in pgbench, the
patch got rejected:-)

An "ignored" tests looked like a good compromise to check how things are
going in the farm and to be able to check for more non regressions when
developing pgbench, without introducing behavioral changes.

> I'm with you on the importance of testing, but it seems better to start
> by making the thing more testable.

I'm used to my test patches being rejected, including modifying pgbench
behavior to make it more testable. Am I mad enough to retry? Maybe, maybe
not.

Attached the fully "ignored" version of the time features test as a patch.

--
Fabien.

Attachment Content-Type Size
pgbench-time-test-1.patch text/x-diff 3.3 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message vignesh C 2021-07-11 13:29:23 Re: Enhanced error message to include hint messages for redundant options error
Previous Message vignesh C 2021-07-11 13:06:19 Re: psql - factor out echo code