Re: pgbench --continue-on-error: clarify TPS and failure reporting

On Thu, Jun 18, 2026 at 3:25 PM Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com> wrote:
> Thanks for confirming the doc issue. The here comes a small doc patch.
>
> I also noticed there is a later paragraph about “latency” that also needs an update, it’s included in the patch as well.

Thanks for the patch! The TPS-related changes look good to me.

The latency of a successful transaction includes the entire time of
- transaction execution with rollbacks and retries. The latency is measured
- only for successful transactions and commands but not for failed
transactions
- or commands.
+ transaction execution with rollbacks and retries. The latency is measured
+ only for successful transactions and commands, except that failed
+ transactions are included in the simple average latency when the
+ <option>--continue-on-error</option> option is used.

It doesn't seem that the behavior of the latency average in the main
report depends on --continue-on-error itself. Instead, when neither
--progress, --rate, nor --latency-limit is specified, the main
report shows a simple latency average computed as the total benchmark
duration divided by the number of successful and failed transactions,
so failed transactions are included.

On the other hand, when any of those options is specified, the
latency average and latency stddev in the main report are based
only on successful transactions. Likewise, the per-script and
per-command latency statistics are measured only for successful
transactions and commands.

So how about something like this instead?

-------------------------------
<para>
The latency of a successful transaction includes the entire time of
- transaction execution with rollbacks and retries. The simple
- <literal>latency average</literal> shown in the main report is computed from
- the total benchmark duration divided by both successful and failed
- transactions, and therefore includes failed transactions. In contrast,
- detailed latency statistics, including the per-script and per-command
- reports, are measured only for successful transactions and commands, not
- for failed transactions or commands.
+ transaction execution with rollbacks and retries. In the main report, when
+ neither <option>--progress</option>, <option>--rate</option>, nor
+ <option>--latency-limit</option> is specified, the
+ <literal>latency average</literal> is computed from the total benchmark
+ duration divided by both successful and failed transactions, and therefore
+ includes failed transactions. Otherwise, the <literal>latency
average</literal>
+ and <literal>latency stddev</literal> shown in the main report are measured
+ only for successful transactions. Detailed latency statistics, including
+ the per-script and per-command reports, are also measured only for
+ successful transactions and commands, not for failed transactions or
+ commands.
</para>
-------------------------------

Regards,

--
Fujii Masao