Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY)

From: Boris Mironov <boris_mironov(at)outlook(dot)com>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY)
Date: 2025-11-11 13:33:16
Message-ID: DS0PR08MB9565D91414C65B3AC363825488CFA@DS0PR08MB9565.namprd08.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello hackers,

For some of my specific hardware tests I needed to generate big databases well beyond RAM size. Hence I turned to pgbench tool and its default 2 modes for client- and server-side generation for TPC-B tests. When I use "scale" factor in range of few thousands (eg, 3000 - 5000) data generation phase takes quite some time. I looked at it as opportunity to prove/disprove 2 hypothesises:

*
will INSERT mode work faster if we commit once every "scale" and turn single INSERT into "for" loop with commits for 3 tables in the end of each loop
*
will "INSERT .. SELECT FROM unnest" be faster than "INSERT .. SELECT FROM generate_series"
*
will BINARY mode work faster than TEXT even though we send much more data
*
and so on

As a result of my experiments I produced significant patch for pgbench utility and though that it might be of interest not just for me. Therefore I'm sending draft version of it in diff format for current development tree on GitHub. As of November 11, 2025 I can merge with main branch of the project on GitHub.

Spoiler alert: "COPY FROM BINARY" is significantly faster than current "COPY FROM TEXT"

Would be happy to polish it if there is interest to such change.

Cheers

Attachment Content-Type Size
pgbench.c.diff application/octet-stream 29.0 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2025-11-11 14:04:12 Re: CURL_IGNORE_DEPRECATION
Previous Message Peter Eisentraut 2025-11-11 13:27:22 Re: [PATCH] Fix msvc_gendef.pl for aarch64 (Windows on Arm)