pgbench --continue-on-error: clarify TPS and failure reporting

From: Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Cc: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, Rintaro Ikeda <ikedarintarof(at)oss(dot)nttdata(dot)com>, Yugo Nagata <nagata(at)sraoss(dot)co(dot)jp>
Subject: pgbench --continue-on-error: clarify TPS and failure reporting
Date: 2026-06-18 03:24:04
Message-ID: 597C9755-4386-488D-A289-16D50AD81FBF@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi

While testing “[0ab208fa5] pgbench: Add --continue-on-error option”, I got a question about the doc:
```
<para>
This option is useful when your custom script may raise errors
such as unique constraint violations, but you want the benchmark
to continue and measure performance including those failures.
</para>
```

The phrase “measure performance including those failures” gives the impression that failed transactions might be counted in TPS, but they are not.

In my test, all transactions failed, and TPS was reported as 0:
```
% pgbench -d evantest -n -t 5 --continue-on-error --failures-detailed -f pgbench-coe.sql
pgbench (19beta1)
transaction type: pgbench-coe.sql
scaling factor: 1
query mode: simple
number of clients: 1
number of threads: 1
maximum number of tries: 1
number of transactions per client: 5
number of transactions actually processed: 0/5
number of failed transactions: 5 (100.000%)
number of serialization failures: 0 (0.000%)
number of deadlock failures: 0 (0.000%)
number of other failures: 5 (100.000%)
latency average = 0.389 ms (including failures)
initial connection time = 4.172 ms
tps = 0.000000 (without initial connection time)
```

If this behavior is intended, I wonder if we should clarify the TPS counting logic more explicitly. For example:
```
<para>
This option is useful when your custom script may raise errors
such as unique constraint violations, but you want the benchmark
to continue despite individual statement failures. Failed transactions
are reported separately and included in latency statistics, but are not
counted as transactions actually processed for TPS.
</para>
```

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alena Rybakina 2026-06-18 03:32:27 Re: Vacuum statistics
Previous Message Amit Kapila 2026-06-18 03:19:53 Re: Fix race in ReplicationSlotRelease for ephemeral slots