| From: | Hannu Krosing <hannuk(at)google(dot)com> |
|---|---|
| To: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | [PATCH] Refactor pgbench to make future improvements easier |
| Date: | 2026-10-03 11:02:47 |
| Message-ID: | CAMT0RQT1BMSKqgO31QJw4razSDruW_HX6G-R_Xr7yXsfMM_H2g@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi hackers,
`src/bin/pgbench/pgbench.c` has grown into a ~8,150-line monolithic source
file combining benchmark statistics, variable management, socket event
multiplexing, script/expression parsing and evaluation, meta-command
handlers, schema/data initialization, and the multi-threaded client state
machine.
This 7-patch series splits `src/bin/pgbench/pgbench.c` into focused,
self-contained compilation units while preserving exact behavior and all
existing tests:
1. `stats.c/h`: Extract `SimpleStats` and `StatsData` structures, latency
and retry accumulation, and formatting helpers.
2. `variable.c/h`: Extract `PgBenchValue`, `Variable`, and `Variables`
management, binary-search variable lookup, type coercion, and SQL query
variable substitution (`assignVariables` / `getQueryParams`).
3. `poller.c/h`: Extract the `socket_set` abstraction and platform-specific
`ppoll(2)` / `select(2)` socket multiplexing implementation.
4. `script.c/h`: Extract built-in script definitions, SQL and backslash
meta-command parsing, Bison/Flex expression parser callbacks, and
runtime expression evaluation (`evaluateExpr`, random distributions,
hashing, and `permute`).
5. `commands.c/h`: Extract runtime meta-command execution (`executeMetaCommand`,
`\set`, `\sleep`, `\if`/`\elif`/`\else`/`\endif`, `\shell`/`\setshell`),
`\gset`/`\aset` result processing, and pipeline command preparation.
6. `init.c/h`: Extract `-i` initialization routines (`runInitSteps`), DDL
table/index/partition creation, client-side and server-side data
generation, and vacuum helpers.
7. `engine.c/h`: Extract the client connection state machine
(`advanceConnectionState`), error/retry handling, worker thread loop
(`threadRun`), progress/log reporting, and final results reporting,
leaving `pgbench.c` (~978 lines) dedicated to CLI option parsing (`main`
and `usage`).
Each step builds cleanly with both Make and Meson and passes all 686 TAP
tests in `src/bin/pgbench`.
This modularization also prepares a clean boundary for extracting pgbench's
random number distributions, hashing, and permutation routines into
`src/common` (`pgbench_funcs.c/h`) and exposing them in SQL via
`contrib/pgbench`.
Hannu Krosing (7):
Refactor pgbench: extract statistics and latency tracking into
stats.c/h
Refactor pgbench: extract scoped variable store and value evaluation
into variable.c/h
Refactor pgbench: extract socket multiplexing and event poller into
poller.c/h
Refactor pgbench: extract script parsing and expression evaluation
into script.c/h
Refactor pgbench: extract meta-command execution and handlers into
commands.c/h
Refactor pgbench: extract initialization, schema, and data generators
into init.c/h
Refactor pgbench: extract execution engine, state machine, and thread
runner into engine.c/h
src/bin/pgbench/Makefile | 9 +-
src/bin/pgbench/commands.c | 737 ++++
src/bin/pgbench/commands.h | 70 +
src/bin/pgbench/engine.c | 2141 +++++++++++
src/bin/pgbench/engine.h | 161 +
src/bin/pgbench/init.c | 786 ++++
src/bin/pgbench/init.h | 66 +
src/bin/pgbench/meson.build | 7 +
src/bin/pgbench/pgbench.c | 7170 +----------------------------------
src/bin/pgbench/pgbench.h | 218 +-
src/bin/pgbench/poller.c | 196 +
src/bin/pgbench/poller.h | 48 +
src/bin/pgbench/script.c | 1973 ++++++++++
src/bin/pgbench/script.h | 253 ++
src/bin/pgbench/stats.c | 203 +
src/bin/pgbench/stats.h | 172 +
src/bin/pgbench/variable.c | 834 ++++
src/bin/pgbench/variable.h | 130 +
18 files changed, 7902 insertions(+), 7272 deletions(-)
create mode 100644 src/bin/pgbench/commands.c
create mode 100644 src/bin/pgbench/commands.h
create mode 100644 src/bin/pgbench/engine.c
create mode 100644 src/bin/pgbench/engine.h
create mode 100644 src/bin/pgbench/init.c
create mode 100644 src/bin/pgbench/init.h
create mode 100644 src/bin/pgbench/poller.c
create mode 100644 src/bin/pgbench/poller.h
create mode 100644 src/bin/pgbench/script.c
create mode 100644 src/bin/pgbench/script.h
create mode 100644 src/bin/pgbench/stats.c
create mode 100644 src/bin/pgbench/stats.h
create mode 100644 src/bin/pgbench/variable.c
create mode 100644 src/bin/pgbench/variable.h
--
2.56.0.rc1.315.gc6ed9934b7-goog
| Attachment | Content-Type | Size |
|---|---|---|
| v2-0001-Refactor-pgbench-extract-statistics-and-latency-t.patch | application/x-patch | 25.2 KB |
| v2-0004-Refactor-pgbench-extract-script-parsing-and-expre.patch | application/x-patch | 260.3 KB |
| v2-0002-Refactor-pgbench-extract-scoped-variable-store-an.patch | application/x-patch | 41.6 KB |
| v2-0007-Refactor-pgbench-extract-execution-engine-state-m.patch | application/x-patch | 138.1 KB |
| v2-0006-Refactor-pgbench-extract-initialization-schema-an.patch | application/x-patch | 49.7 KB |
| v2-0003-Refactor-pgbench-extract-socket-multiplexing-and-.patch | application/x-patch | 13.8 KB |
| v2-0005-Refactor-pgbench-extract-meta-command-execution-a.patch | application/x-patch | 43.6 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Alvaro Herrera | 2026-10-03 11:35:14 | Re: REPACK (CONCURRENTLY) might keep dropped-column data |
| Previous Message | Nitin Motiani | 2026-10-03 10:20:38 | Re: [PATCH v1] Fix for Bug#19724 - ALTER TYPE ... ALTER ATTRIBUTE triggers internal error for base type of domain with check |