[PATCH v1] Free oversized decoded WAL records in XLogReaderFree()

From: Tomás Senart <ts(at)perfloop(dot)ai>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Cc: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Subject: [PATCH v1] Free oversized decoded WAL records in XLogReaderFree()
Date: 2026-11-02 23:15:13
Message-ID: CAEEBa3Uai8TGPd6P5SX_fzbu3KHMwSvu9WuqA8JPSPQ1RJR8fQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

XLogReaderFree() does not free decoded records that were too large for
the decode buffer. XLogReadRecordAlloc() allocates those separately,
and only XLogReleasePreviousRecord() and ResetDecoder() free them. The
record last returned by XLogReadRecord() stays in the decode queue until
the next read, so if it is larger than the decode buffer (64kB by
default), it leaks when the reader is freed.

Most callers free that memory soon after anyway. The checkpointer does
not. CheckPointTwoPhase() calls XlogReadTwoPhaseData() for each
prepared transaction whose PREPARE record is before the redo pointer,
and that function creates and frees a reader in checkpointer_context,
which is reset only after an error. So each large PREPARE record read
there stays allocated until the checkpointer exits.

The attached SQL file reproduces it. It prepares five transactions
with 20000 subtransactions each (80220-byte PREPARE records), runs
CHECKPOINT after each one, and logs the checkpointer's memory contexts.
"Grand total ... used" on master 8628df8891a:

step master patched
1 331136 248392
2 413880 248392
3 496624 248392
4 579368 248392
5 662112 248392

That is 82744 bytes per transaction without the patch. 15.19 shows the
same growth.

The patch calls ResetDecoder() in XLogReaderFree() before the decode
buffer is freed. ResetDecoder() frees only the separately allocated
records, so a decode buffer supplied by the caller is not touched. The
leak goes back to 3f1ce97346 in 15, so I think it should be
backpatched.

With assertions enabled, the regress, recovery, pg_waldump, pg_rewind,
pg_walinspect, test_decoding and pg_walsummary suites give the same
results with and without the patch (the injection-point tests were
skipped).

The leak was found and fixed by Perfloop. I am the contact.
Feel free to rework the patch however you prefer.

Thanks,
Tomás

Attachment Content-Type Size
v1-0001-Free-oversized-decoded-WAL-records-in-XLogReaderF.patch text/x-patch 2.4 KB
xlogreader-leak-repro.sql application/sql 1.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2026-11-02 23:41:30 Re: Parallel vacuum: workers together with leader could overrun cost budget
Previous Message surya poondla 2026-11-02 23:03:59 Re: Fix trigger memory leak