XLog size reductions: smaller XLRec block header for PG17

From: Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Robert Haas <robertmhaas(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Subject: XLog size reductions: smaller XLRec block header for PG17
Date: 2023-05-18 14:59:37
Message-ID: CAEze2WjuJqVeB6EUZ1z75_ittk54H6Lk7WtwRskEeGtZubr4bQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi all,

As was previously discussed at the thread surrounding [1]: Currently
any block registration in WAL takes up at least 8 bytes in xlog
overhead, regardless of how much data is included in that block: 1
byte for block ID, 1 byte for fork and flags, 2 bytes for the block
data length, and 4 bytes for the blockNo. (Usually, another 12 bytes
are used for a RelFileLocator; but that may not be included for some
blocks in the record when conditions apply)

Attached is a patch that reduces this overhead by up to 2 bytes by
encoding how large the block data length field is into the block ID,
and thus optionally reducing the block data's length field to 0 bytes.
Examples: cross-page update records will now be 2 bytes shorter,
because the record never registers any data for the new block of the
update; pgbench transactions are now either 6 or 8 bytes smaller
depending on whether the update crosses a page boundary (in xlog
record size; after alignment it is 0 or 4/8 bytes, depending on
MAXALIGN and whether the updates are cross-page updates).

It changes the block IDs used to fit in 6 bits, using the upper 2 bits
of the block_id field to store how much data is contained in the
record (0, <=UINT8_MAX, or <=UINT16_MAX bytes).

This is part 1 of a series of patches trying to decrease the size of
WAL; see also [0], [1] and [2] for more info on what's still to go.
I'm working on a separate, much more involved patch for the XLogRecord
header itself, similar to the patch in [1], which I expect to send
sometime soon as well.
Unless someone thinks the patches should be discussed as one series,
I'm planning on posting that in another thread, as I don't see any
meaningful dependencies between the patches, and the XLR header patch
will be quite a bit larger than this one.

Kind regards,

Matthias van de Meent
Neon, Inc.

[0] https://wiki.postgresql.org/wiki/Updating_the_WAL_infrastructure
[1] https://www.postgresql.org/message-id/flat/CAEze2Wjd3jY_UhhOGdGGnC6NO%3D%2BNmtNOmd%3DJaYv-v-nwBAiXXA%40mail.gmail.com#17a51d83923f4390d8f407d0d6c5da07
[2] https://www.postgresql.org/message-id/flat/CAEze2Whf%3DfwAj7rosf6aDM9t%2B7MU1w-bJn28HFWYGkz%2Bics-hg%40mail.gmail.com

PS. Benchmark results on my system (5950x with other light tasks
running) don't show an obviously negative effect in a 10-minute run
with these arbitrary pgbench settings on a fresh cluster with default
configuration:

./pg_install/bin/pgbench postgres -j 2 -c 6 -T 600 -M prepared
[...]
master: tps = 375
patched: tps = 381

Attachment Content-Type Size
v1-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch application/octet-stream 10.9 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Nathan Bossart 2023-05-18 15:20:18 Re: The documentation for READ COMMITTED may be incomplete or wrong
Previous Message Tom Lane 2023-05-18 14:53:35 Re: The documentation for READ COMMITTED may be incomplete or wrong