Add the ability to limit the amount of memory that can be allocated to backends.

From: Reid Thompson <reid(dot)thompson(at)crunchydata(dot)com>
To: pgsql-hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: reid(dot)thompson(at)crunchydata(dot)com
Subject: Add the ability to limit the amount of memory that can be allocated to backends.
Date: 2022-08-31 16:50:19
Message-ID: bd57d9a4c219cc1392665fd5fba61dde8027b3da.camel@crunchydata.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Hackers,

Add the ability to limit the amount of memory that can be allocated to
backends.

This builds on the work that adds backend memory allocated to
pg_stat_activity
https://www.postgresql.org/message-id/67bb5c15c0489cb499723b0340f16e10c22485ec.camel%40crunchydata.com
Both patches are attached.

Add GUC variable max_total_backend_memory.

Specifies a limit to the amount of memory (MB) that may be allocated to
backends in total (i.e. this is not a per user or per backend limit).
If unset, or set to 0 it is disabled. It is intended as a resource to
help avoid the OOM killer. A backend request that would push the total
over the limit will be denied with an out of memory error causing that
backends current query/transaction to fail. Due to the dynamic nature
of memory allocations, this limit is not exact. If within 1.5MB of the
limit and two backends request 1MB each at the same time both may be
allocated exceeding the limit. Further requests will not be allocated
until dropping below the limit. Keep this in mind when setting this
value to avoid the OOM killer. Currently, this limit does not affect
auxiliary backend processes, this list of non-affected backend
processes is open for discussion as to what should/should not be
included. Backend memory allocations are displayed in the
pg_stat_activity view.

--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.

reid(dot)thompson(at)crunchydata(dot)com
www.crunchydata.com

Attachment Content-Type Size
001-dev-max-memory.patch text/x-patch 11.9 KB
001-pg-stat-activity-backend-memory-allocated.patch text/x-patch 26.9 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Pavel Stehule 2022-08-31 16:59:23 Re: [PATCH] Query Jumbling for CALL and SET utility statements
Previous Message Jonathan S. Katz 2022-08-31 16:48:52 Re: SQL/JSON features for v15