Re: BUG #19890: pg_receivewal advances past a truncated LZ4 WAL segment

From: Raja Sai pranav <rajasaipranav0(at)gmail(dot)com>
To: pgsql-bugs(at)lists(dot)postgresql(dot)org
Cc: idrissaajalloh(at)gmail(dot)com
Subject: Re: BUG #19890: pg_receivewal advances past a truncated LZ4 WAL segment
Date: 2026-11-21 06:31:25
Message-ID: nh-8a1537cf-5adc-4bc3-9c2d-da7d258b0543@google.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi,

I reproduced this on master (4a26e36f87) and attached a patch.

FindStreamingStart() decompresses each completed .lz4 segment and checks
only that it yields exactly one segment of data. It never checks that the
LZ4 frame ended. LZ4F_decompress() returns 0 only once a frame is fully
decoded, so the patch records whether the last call returned 0. If it
didn't, the segment is skipped with a warning ("compressed segment file ...
is truncated, skipping"), the same way a segment of the wrong size is
skipped today.

The patch adds a test to src/bin/pg_basebackup/t/
https://www.google.com/url?q=http://020_pg_receivewal.pl&source=gmail&ust=1795329083836000&sa=E.
It copies a completed .lz4 segment into a new directory, cuts off the
4-byte end mark of the frame, and puts a full-size uncompressed segment
just before it, as your script does. pg_receivewal must then warn and
restart at the start of the cut segment. Without the fix it restarts at the
next segment and the test fails; with it, the pg_basebackup TAP suite
passes. The patch applies to REL_18_STABLE and REL_19_STABLE.

Regards,
Sai Pranav

On Sat, Nov 21, 2026 12:00 PM, PG Bug reporting form <noreply(at)postgresql(dot)org>
wrote:

> The following bug has been logged on the website:
>
> Bug reference: 19890
> Logged by: Wren
> Email address: idrissaajalloh(at)gmail(dot)com
> PostgreSQL version: 18.6
> Operating system: Linux/amd64
> Description:
>
> pg_receivewal treats an incomplete LZ4 frame with a completed WAL filename
> as a complete segment when decompression has already produced exactly
> WalSegSz bytes. On restart, it advances to the following segment, skipping
> validation or reacquisition of the damaged segment.
>
> A completed-name LZ4 segment should be accepted only if decompression
> produces exactly WalSegSz bytes and reaches the LZ4 frame-completion state.
> A frame truncated after its final output bytes but before its footer is
> complete should be rejected or skipped.
>
> How to repeat:
>
> Save the following as run.sh in an empty directory. Run it as `sh run.sh
> /absolute/path/to/postgresql-source`. The source tree needs the normal
> build
> prerequisites and the LZ4 development library.
>
> #!/bin/sh
> set -eu
>
> if test "$#" -ne 1; then
> echo "usage: $0 /absolute/path/to/postgresql-source" >&2
> exit 2
> fi
> src=$1
> case "$src" in /*) ;; *) echo "source path must be absolute" >&2; exit 2;;
> esac
> test -x "$src/configure" || { echo "not a PostgreSQL source tree: $src"
> >&2;
> exit 2; }
>
> base=$(pwd)/work.$$
> build=$base/build
> inst=$base/install
> data=$base/data
> sock=$base/socket
> wal=$base/wal
> mkdir -p "$build" "$inst" "$data" "$sock" "$wal"
>
> cd "$build"
> "$src/configure" --prefix="$inst" --with-lz4
> make -j2
> make install
>
> "$inst/bin/initdb" -D "$data" -A trust --no-locale
> cat >>"$data/postgresql.conf" <<EOF
> listen_addresses = ''
> unix_socket_directories = '$sock'
> wal_level = replica
> max_wal_senders = 5
> EOF
> "$inst/bin/pg_ctl" -D "$data" -l "$base/server.log" start
> trap '"$inst/bin/pg_ctl" -D "$data" stop -m immediate >/dev/null 2>&1 ||
> true' EXIT HUP INT TERM
>
> export PGHOST=$sock PGDATABASE=postgres
> # Receive two whole segments. Repeated switches are used because the first
> # switch can finish the segment in which initdb/startup left the server.
> endpos=$("$inst/bin/psql" -AtX -c "SELECT '0/0'::pg_lsn +
> ((((pg_current_wal_insert_lsn()-'0/0')::bigint/16777216)+2)*16777216)")
> (
> sleep 1
> "$inst/bin/psql" -X -c 'SELECT pg_switch_wal()' >/dev/null
> "$inst/bin/psql" -X -c 'CREATE TABLE lz4_frame_probe(a text);
> INSERT
> INTO lz4_frame_probe SELECT repeat(md5(g::text),100) FROM
> generate_series(1,20000) g; SELECT pg_switch_wal(); DROP TABLE
> lz4_frame_probe; SELECT pg_switch_wal()' >/dev/null
> ) &
> producer=$!
> "$inst/bin/pg_receivewal" -D "$wal" -Z lz4 -E "$endpos" -n -v
> >"$base/first.log" 2>&1
> wait "$producer"
>
> set -- "$wal"/????????????????????????.lz4
> test "$#" -ge 2 || { echo "SETUP ERROR: fewer than two completed LZ4
> segments"; exit 2; }
> highest=
> previous=
> for f in $(printf '%s\n' "$wal"/????????????????????????.lz4 | sort); do
> previous=$highest; highest=$f; done
> test -n "$previous" && test -n "$highest"
> before=$(wc -c <"$highest")
> truncate -s $((before - 1)) "$highest"
>
> # With the lower complete segment retained, rejection has an unambiguous
> # oracle: restart at the corrupted segment, not its successor.
> set +e
> "$inst/bin/pg_receivewal" -D "$wal" -Z lz4 -E 0/1 -n -v >"$base/second.log"
> 2>&1
> rc=$?
> set -e
> cat "$base/second.log"
>
> highhex=$(basename "$highest" .lz4)
> logid=$(printf '%d' "0x$(printf '%s' "$highhex" | cut -c9-16)")
> segid=$(printf '%d' "0x$(printf '%s' "$highhex" | cut -c17-24)")
> nextno=$((logid * 256 + segid + 1))
> ptr_hi=$((nextno / 256))
> ptr_lo=$(((nextno % 256) * 16777216))
> successor=$(printf '%X/%08X' "$ptr_hi" "$ptr_lo")
> if grep -q "starting log streaming at $successor" "$base/second.log"; then
> echo "BUG REPRODUCED: truncated $(basename "$highest") was
> accepted;
> rc=$rc"
> exit 1
> fi
> echo "NOT REPRODUCED: truncated frame was not used as a completed segment;
> rc=$rc"
> exit 0
>
> The affected behavior is reported when the second pg_receivewal invocation
> starts at the successor of the truncated highest segment and the script
> prints:
>
> BUG REPRODUCED: truncated <WAL filename>.lz4 was accepted; rc=0
>
> The script then exits with status 1. Exit status 2 indicates a setup
> failure. A corrected implementation should reject or skip the truncated
> frame, avoid starting at its successor, print `NOT REPRODUCED`, and exit
> with status 0.
>
> In the recorded PostgreSQL 18.6 run, a completed segment was truncated by
> one byte and pg_receivewal produced:
>
> TRUNCATED_FILE=00000001000000000000001A.lz4 BEFORE=140130 AFTER=140129
> pg_receivewal: starting log streaming at 0/1B000000 (timeline 1)
> pg_receivewal: stopped log streaming at 0/1B000000 (timeline 1)
> SECOND_RC=0
>
> Here 0/1B000000 is the boundary following the truncated
> 00000001000000000000001A.lz4 segment, showing that directory scanning
> classified the incomplete frame as complete.
>
> The issue was reproduced on PostgreSQL 18.6 on Linux/amd64 with LZ4
> support.
> The LZ4 path used to find the streaming start counts decompressed bytes but
> does not require LZ4F_decompress() to return 0, the frame-completion
> signal.
> Consequently, truncating the frame footer can leave the decompressed size
> equal to WalSegSz and allow the file to be accepted.
>
> This occurs when pg_receivewal restarts from a directory whose highest
> completed-name LZ4 WAL file is truncated after producing a full segment but
> before frame completion.
>
>
>
>
>
>
>

Attachment Content-Type Size
v1-0001-pg_receivewal-Skip-a-completed-LZ4-segment-whose-.patch text/x-patch 4.4 KB

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Raja Sai pranav 2026-11-21 06:42:40 Re: BUG #19874: CREATE SCHEMA view binds same-named table in pg_temp and fails
Previous Message Raja Sai pranav 2026-11-21 06:24:20 Re: BUG #19881: CLUSTER uses a disabled sort path when enable_sort is off