Re: Trim the heap free memory

From: shawn wang <shawn(dot)wang(dot)pg(at)gmail(dot)com>
To: Tomas Vondra <tomas(at)vondra(dot)me>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, David Rowley <dgrowleyml(at)gmail(dot)com>, Rafia Sabih <rafia(dot)pghackers(at)gmail(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Trim the heap free memory
Date: 2025-07-05 17:47:08
Message-ID: CA+T=_GUddKqkiVHHj1NgZ3htqKo317rHPHSPdnY=XTGA6f8HVA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thank you very much for your response.

> To propose something less abstract / more tangible, I think we should do
> something like this:
>
> 1) add a bit of code for glibc-based systems, that adjusts selected
> malloc parameters using mallopt() during startup
>
> 2) add a GUC that enables this, with the default being the regular glibc
> behavior (with dynamic adjustment of various thresholds)
>

I believe that the issue here arises from design incompatibilities between
the complex engineering code and ptmalloc.
Modifying malloc parameters through mallopt is not user-friendly for
database users and can be overly complex.
Moreover, setting certain parameters may lead to performance issues.
Monitoring memory usage should be a common practice for all database users.
With my signal-based approach, we can trigger a trim operation
when high memory usage is detected or by setting up a scheduled task.
This reduces the complexity for users and also helps in lowering memory
consumption.
Of course, this solution is not perfect and does not address the problem
elegantly from a fundamental perspective.
However, it has proven effective in the user environment.
I have set up a scheduled task to execute a function every 10 minutes for
processes exceeding 50MB.
This has reduced memory usage from 87% to 30% on a 64GB system.

Best regards

Shawn

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2025-07-05 18:00:07 Re: A assert failure when initdb with track_commit_timestamp=on
Previous Message Tom Lane 2025-07-05 17:34:13 Re: [PATCH] initdb: Treat empty -U argument as unset username