| From: | "Zhou, Zhiguo" <zhiguo(dot)zhou(at)intel(dot)com> |
|---|---|
| To: | Yura Sokolov <y(dot)sokolov(at)postgrespro(dot)ru>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Cc: | "Li, Tianyou" <tianyou(dot)li(at)intel(dot)com>, "Kim, Andrew" <andrew(dot)kim(at)intel(dot)com> |
| Subject: | Re: [RFC] Lock-free XLog Reservation from WAL |
| Date: | 2026-11-19 15:21:02 |
| Message-ID: | CY8PR11MB78985783B939B65B10352649EF792@CY8PR11MB7898.namprd11.prod.outlook.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi all,
I would like to revisit this work with CAS128, responding to Yura's
question about whether the predecessor hash table justified its
complexity. Thanks to Yura for that work, and to Japin, Andres and others
for the measurements and counterexamples.
Attached is the v8 CAS128 patch for CF 5633, based on master 4280abf.
It updates CurrBytePos and PrevBytePos together, keeping the full
xl_prev and the WAL format. The required primitive lives in
port/atomics; WAL keeps its existing spinlock path when native CAS128
is unavailable. No predecessor hash table is needed.
This patch retains the compile-time default of 8 insertion locks. The
wal_insert_locks GUC patch is being submitted separately in its existing
thread [1], tracked by CF 6193. Each patch applies independently to
4280abf, so each CF entry can test its own attachment.
On x86-64, the native path needs a CPU with cx16 and a compiler target
that enables it, such as -mcx16. Autoconf and Meson check for native
support and otherwise select the existing spinlock path; selection is
at build time. The CAS operand has alignas(16), with XLogCtl's
cache-line-aligned allocation providing the required shared-memory
alignment. Assertions also check this alignment.
Andrew Kim reviewed both patches off-list on 5ffc821: build checks,
atomics integration, alignment and GUC validation. He built the native
and fallback paths and independently benchmarked the series. Both
patches carry his Reviewed-by and Tested-by trailers.
His HammerDB TPROC-C results use master 5ffc821 and Intel Xeon CPUs,
2 sockets / 384 threads. The native CAS builds use GCC 14.2 and
CFLAGS='-O2 -mcx16'. The workload is 250 warehouses / 256 VUs, 2 minutes
ramp-up and 3 minutes measurement, with 3 mirrored blocks. WAL was on
tmpfs, with no checkpoints inside the measurement windows. Changes in
mean NOPM relative to unmodified master8 were below. The GUC rows include
the separately proposed wal_insert_locks patch:
Build Locks vs master8
----------------------------------- ----- ----------
Master, unmodified 8 0.00%
CAS128 + GUC, default 8 -0.2%
CAS128 alone, compile-time locks 64 +33.7%
CAS128 + GUC 64 +40.4%
Default-8 throughput stays close to master in this HammerDB workload;
the larger gains come with CAS128 and 64 insertion locks.
Earlier same-64-lock HammerDB comparisons, including Andrew's previous
campaign, showed gains comparable to v7 hash with less predecessor
bookkeeping. We also reproduced CAS128 gains on several other
high-core-count Xeon platforms. Those earlier measurements used
f58ab18/b0392c2-era code.
On 5ffc821, our Linux native/fallback Autoconf/Meson and CAS-only
EXEC_BACKEND checks passed core regressions and recovery/GUC tests.
Windows, ARM and big-endian execution remain unverified.
Increasing the locks can still regress Andres's 1 MiB
pg_logical_emit_message workload even with CAS128. A separate writer
diagnostic matched the insertion-completion wait to the batch actually
written; it is not included here and needs separate discussion.
Autoconf LLVM bitcode flags still need manual matching when native CAS
is enabled; details are in the accompanying benchmark note.
I would appreciate review of the paired-position CAS, the minimal
atomics interface and the native/fallback build checks. Discussion of
the insertion-lock setting can continue in its own thread [1].
Regards,
Zhiguo
[1] https://postgr.es/m/flat/3b11fdc2-9793-403d-b3d4-67ff9a00d447(at)postgrespro(dot)ru
________________________________________
From: Zhou, Zhiguo <zhiguo(dot)zhou(at)intel(dot)com>
Sent: Friday, October 31, 2025 7:19 PM
To: Yura Sokolov <y(dot)sokolov(at)postgrespro(dot)ru>; pgsql-hackers(at)lists(dot)postgresql(dot)org <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [RFC] Lock-free XLog Reservation from WAL
Rebase again.
Regards,
Zhiguo
On 4/30/2025 10:55 PM, Yura Sokolov wrote:
> Just rebase
>
| Attachment | Content-Type | Size |
|---|---|---|
| v8-0001-cas128-atomics-4280abf.patch | application/octet-stream | 33.5 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Nathan Bossart | 2026-11-19 15:21:49 | Re: Autovacuum freeze scores wrap around for just-vacuumed tables |
| Previous Message | Tom Lane | 2026-11-19 15:05:14 | Re: Block various aggregate component functions from being called directly |