[PATCH] Refactor pgbench to make future improvements easier

From: Hannu Krosing <hannuk(at)google(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: [PATCH] Refactor pgbench to make future improvements easier
Date: 2026-10-03 11:02:47
Message-ID: CAMT0RQT1BMSKqgO31QJw4razSDruW_HX6G-R_Xr7yXsfMM_H2g@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

`src/bin/pgbench/pgbench.c` has grown into a ~8,150-line monolithic source
file combining benchmark statistics, variable management, socket event
multiplexing, script/expression parsing and evaluation, meta-command
handlers, schema/data initialization, and the multi-threaded client state
machine.

This 7-patch series splits `src/bin/pgbench/pgbench.c` into focused,
self-contained compilation units while preserving exact behavior and all
existing tests:

1. `stats.c/h`: Extract `SimpleStats` and `StatsData` structures, latency
and retry accumulation, and formatting helpers.
2. `variable.c/h`: Extract `PgBenchValue`, `Variable`, and `Variables`
management, binary-search variable lookup, type coercion, and SQL query
variable substitution (`assignVariables` / `getQueryParams`).
3. `poller.c/h`: Extract the `socket_set` abstraction and platform-specific
`ppoll(2)` / `select(2)` socket multiplexing implementation.
4. `script.c/h`: Extract built-in script definitions, SQL and backslash
meta-command parsing, Bison/Flex expression parser callbacks, and
runtime expression evaluation (`evaluateExpr`, random distributions,
hashing, and `permute`).
5. `commands.c/h`: Extract runtime meta-command execution (`executeMetaCommand`,
`\set`, `\sleep`, `\if`/`\elif`/`\else`/`\endif`, `\shell`/`\setshell`),
`\gset`/`\aset` result processing, and pipeline command preparation.
6. `init.c/h`: Extract `-i` initialization routines (`runInitSteps`), DDL
table/index/partition creation, client-side and server-side data
generation, and vacuum helpers.
7. `engine.c/h`: Extract the client connection state machine
(`advanceConnectionState`), error/retry handling, worker thread loop
(`threadRun`), progress/log reporting, and final results reporting,
leaving `pgbench.c` (~978 lines) dedicated to CLI option parsing (`main`
and `usage`).

Each step builds cleanly with both Make and Meson and passes all 686 TAP
tests in `src/bin/pgbench`.

This modularization also prepares a clean boundary for extracting pgbench's
random number distributions, hashing, and permutation routines into
`src/common` (`pgbench_funcs.c/h`) and exposing them in SQL via
`contrib/pgbench`.

Hannu Krosing (7):
Refactor pgbench: extract statistics and latency tracking into
stats.c/h
Refactor pgbench: extract scoped variable store and value evaluation
into variable.c/h
Refactor pgbench: extract socket multiplexing and event poller into
poller.c/h
Refactor pgbench: extract script parsing and expression evaluation
into script.c/h
Refactor pgbench: extract meta-command execution and handlers into
commands.c/h
Refactor pgbench: extract initialization, schema, and data generators
into init.c/h
Refactor pgbench: extract execution engine, state machine, and thread
runner into engine.c/h

src/bin/pgbench/Makefile | 9 +-
src/bin/pgbench/commands.c | 737 ++++
src/bin/pgbench/commands.h | 70 +
src/bin/pgbench/engine.c | 2141 +++++++++++
src/bin/pgbench/engine.h | 161 +
src/bin/pgbench/init.c | 786 ++++
src/bin/pgbench/init.h | 66 +
src/bin/pgbench/meson.build | 7 +
src/bin/pgbench/pgbench.c | 7170 +----------------------------------
src/bin/pgbench/pgbench.h | 218 +-
src/bin/pgbench/poller.c | 196 +
src/bin/pgbench/poller.h | 48 +
src/bin/pgbench/script.c | 1973 ++++++++++
src/bin/pgbench/script.h | 253 ++
src/bin/pgbench/stats.c | 203 +
src/bin/pgbench/stats.h | 172 +
src/bin/pgbench/variable.c | 834 ++++
src/bin/pgbench/variable.h | 130 +
18 files changed, 7902 insertions(+), 7272 deletions(-)
create mode 100644 src/bin/pgbench/commands.c
create mode 100644 src/bin/pgbench/commands.h
create mode 100644 src/bin/pgbench/engine.c
create mode 100644 src/bin/pgbench/engine.h
create mode 100644 src/bin/pgbench/init.c
create mode 100644 src/bin/pgbench/init.h
create mode 100644 src/bin/pgbench/poller.c
create mode 100644 src/bin/pgbench/poller.h
create mode 100644 src/bin/pgbench/script.c
create mode 100644 src/bin/pgbench/script.h
create mode 100644 src/bin/pgbench/stats.c
create mode 100644 src/bin/pgbench/stats.h
create mode 100644 src/bin/pgbench/variable.c
create mode 100644 src/bin/pgbench/variable.h

--
2.56.0.rc1.315.gc6ed9934b7-goog

Attachment Content-Type Size
v2-0001-Refactor-pgbench-extract-statistics-and-latency-t.patch application/x-patch 25.2 KB
v2-0004-Refactor-pgbench-extract-script-parsing-and-expre.patch application/x-patch 260.3 KB
v2-0002-Refactor-pgbench-extract-scoped-variable-store-an.patch application/x-patch 41.6 KB
v2-0007-Refactor-pgbench-extract-execution-engine-state-m.patch application/x-patch 138.1 KB
v2-0006-Refactor-pgbench-extract-initialization-schema-an.patch application/x-patch 49.7 KB
v2-0003-Refactor-pgbench-extract-socket-multiplexing-and-.patch application/x-patch 13.8 KB
v2-0005-Refactor-pgbench-extract-meta-command-execution-a.patch application/x-patch 43.6 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2026-10-03 11:35:14 Re: REPACK (CONCURRENTLY) might keep dropped-column data
Previous Message Nitin Motiani 2026-10-03 10:20:38 Re: [PATCH v1] Fix for Bug#19724 - ALTER TYPE ... ALTER ATTRIBUTE triggers internal error for base type of domain with check