Re: Protect syscache from bloating with negative cache entries

From: Vladimir Sitnikov <sitnikov(dot)vladimir(at)gmail(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>, Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, "andres(at)anarazel(dot)de" <andres(at)anarazel(dot)de>, "alvherre(at)2ndquadrant(dot)com" <alvherre(at)2ndquadrant(dot)com>, "tomas(dot)vondra(at)2ndquadrant(dot)com" <tomas(dot)vondra(at)2ndquadrant(dot)com>, "bruce(at)momjian(dot)us" <bruce(at)momjian(dot)us>, "tgl(at)sss(dot)pgh(dot)pa(dot)us" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "Ideriha, Takeshi" <ideriha(dot)takeshi(at)jp(dot)fujitsu(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, "michael(dot)paquier(at)gmail(dot)com" <michael(dot)paquier(at)gmail(dot)com>, "david(at)pgmasters(dot)net" <david(at)pgmasters(dot)net>, "craig(at)2ndquadrant(dot)com" <craig(at)2ndquadrant(dot)com>
Subject: Re: Protect syscache from bloating with negative cache entries
Date: 2019-03-01 05:19:22
Message-ID: CAB=Je-FCZBfS+TjwDJEeCtVaas++_tvFRyHJdYLdcgEQf78BjQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Robert> This email thread is really short on clear demonstrations that X or Y
Robert> is useful.

It is useful when the whole database does **not** crash, isn't it?

Case A (==current PostgeSQL mode): syscache grows, then OOMkiller
chimes in, kills the database process, and it leads to the complete
cluster failure (all other PG processes terminate themselves).

Case B (==limit syscache by 10MiB or whatever as Tsunakawa, Takayuki
asks): a single ill-behaved process works a bit slower and/or
consumers more CPU than the other ones. The whole DB is still alive.

I'm quite sure "case B" is much better for the end users and for the
database administrators.

So, +1 to Tsunakawa, Takayuki, it would be so great if there was a way
to limit the memory consumption of a single process (e.g. syscache,
workmem, etc, etc).

Robert> However, memory usage is quite unpredictable. It depends on how many
Robert> backends are active

The number of backends can be limited by ensuring a proper limits at
application connection pool level and/or pgbouncer and/or things like
that.

Robert>how many copies of work_mem and/or
Robert> maintenance_work_mem are in use

There might be other patches to cap the total use of
work_mem/maintenance_work_mem,

Robert>I don't think we
Robert> can say that just imposing a limit on the size of the system caches is
Robert> going to be enough to reliably prevent an out of memory condition

The less possibilities there are for OOM the better. Quite often it is
much better to fail a single SQL rather than kill all the DB
processes.

Vladimir

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David G. Johnston 2019-03-01 05:27:48 Re: Remove Deprecated Exclusive Backup Mode
Previous Message Tatsuro Yamada 2019-03-01 05:17:51 Re: [HACKERS] CLUSTER command progress monitor