Avoid memory leaks during base backups

From: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Avoid memory leaks during base backups
Date: 2022-09-26 13:36:35
Message-ID: CALj2ACXqvfKF2B0beQ=aJMdWnpNohmBPsRg=EDQj_6y1t2O8mQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Postgres currently can leak memory if a failure occurs during base
backup in do_pg_backup_start() or do_pg_backup_stop() or
perform_base_backup(). The palloc'd memory such as backup_state or
tablespace_map in xlogfuncs.c or basebackup.c or tablespaceinfo or the
memory that gets allocated by bbsink_begin_backup() in
perform_base_backup() or any other, is left-out which may cause memory
bloat on the server eventually. To experience this issue, run
pg_basebackup with --label name longer than 1024 characters and
observe memory with watch command, the memory usage goes up.

It looks like the memory leak issue has been there for quite some
time, discussed in [1].

I'm proposing a patch that leverages the error callback mechanism and
memory context. The design of the patch is as follows:
1) pg_backup_start() and pg_backup_stop() - the error callback frees
up the backup_state and tablespace_map variables allocated in
TopMemoryContext. We don't need a separate memory context here because
do_pg_backup_start() and do_pg_backup_stop() don't return any
dynamically created memory for now. We can choose to create a separate
memory context for the future changes that may come, but now it is not
required.
2) perform_base_backup() - a new memory context has been created that
gets deleted by the callback upon error.

The error callbacks are typically called for all the elevels, but we
need to free up the memory only when elevel is >= ERROR or ==
COMMERROR. The COMMERROR is a common scenario because the server can
close the connection to the client or vice versa in which case the
base backup fails. For all other elevels like WARNING, NOTICE, DEBUGX,
INFO etc. we don't free up the memory.

I'm attaching v1 patch herewith.

Thoughts?

[1] https://www.postgresql.org/message-id/Yyq15ekNzjZecwMW%40paquier.xyz

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachment Content-Type Size
v1-0001-Avoid-memory-leaks-during-base-backups.patch application/octet-stream 8.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Simon Riggs 2022-09-26 13:57:04 Re: SUBTRANS: Minimizing calls to SubTransSetParent()
Previous Message Drouvot, Bertrand 2022-09-26 13:29:39 Re: SYSTEM_USER reserved word implementation