Re: Suggestion to add --continue-client-on-abort option to pgbench

On Tue, 17 Jun 2025 16:28:28 +0900
Yugo Nagata <nagata(at)sraoss(dot)co(dot)jp> wrote:

> On Tue, 17 Jun 2025 03:47:00 +0000
> "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com> wrote:
>
> > Dear Nagata-san,
> >
> > > > > 2. The exit-on-abort option and continue-on-error option are mutually
> > > exclusive.
> > > > > Therefore, I've updated the patch to throw a FATAL error when two options
> > > are
> > > > > set simultaneously. Corresponding explanation was also added.
> > >
> > > I don't think that's right since "abort" and "error" are different concept in pgbench.
> > > (Here, "abort" refers to the termination of a client, not a transaction abort.)
> > >
> > > The --exit-on-abort option forces to exit pgbench immediately when any client is
> > > aborted
> > > due to some error. When the --continue-on-error option is not set, SQL errors
> > > other than
> > > deadlock or serialization error cause a client to be aborted. On the other hand,
> > > when the option
> > > is set, clients are not aborted due to any SQL errors; instead they continue to run
> > > after them.
> > > However, clients can still be aborted for other reasons, such as connection
> > > failures or
> > > meta-command errors (e.g., \set x 1/0). In these cases, the --exit-on-abort option
> > > remains
> > > useful even when --continue-on-error is enabled.
> >
> > To clarify: another approach is that allow --continue-on-error option to continue
> > running even when clients meet such errors. Which one is better?
>
> It might be worth discussing which types of errors this option should allow pgbench
> to continue after. On my understand the current patch's goal is to allow only SQL
> level errors like comstraint violations. It seems good because this could simulate
> behaviour of applications that ignore or retry such errors (although they are not
> retried in the current patch). Perhaps, it makes sense to allow to continue after
> some network errors because it would enable benchmarks usign a cluster system as a
> cloud service that could report a temporary error during a failover.

I apologize for accidentally leaving the draft paragraph just above in my previous post.
Please ignore it.

> It might be worth discussing which types of errors this option should allow pgbench to
> continue after.
>
> As I understand it, the current patch aims to allow continuation only after SQL-level
> errors, such as constraint violations. That seems reasonable, as it can simulate the
> behavior of applications that ignore or retry such errors (even though retries are not
> implemented in the current patch).
>
> Perhaps it also makes sense to allow continuation after certain network errors, as this
> would enable benchmarking with cluster systems or cloud services, which might report
> temporary errors during a failover. We would need additional work to properly detect
> and handle network errors, though.
>
> However, I'm not sure it's reasonable to allow continuation after other types of errors,
> such as misuse of meta-commands or unexpected errors during their execution, since these
> wouldn't simulate any real application behavior and would more likely indicate a failure
> in the benchmarking process itself.
>
> Best regards,
> Yugo Nagata
>
> --
> Yugo Nagata <nagata(at)sraoss(dot)co(dot)jp>
>
>

--
Yugo Nagata <nagata(at)sraoss(dot)co(dot)jp>