| From: | Jeremy Schneider <schneider(at)ardentperf(dot)com> |
|---|---|
| To: | "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Cc: | Marat Buharov <marat(dot)buharov(at)gmail(dot)com> |
| Subject: | client_connection_check_interval default value |
| Date: | 2026-02-05 05:30:32 |
| Message-ID: | 20260204213032.15bab46b@ardentperf.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
What would people here think about changing the default value of
client_connection_check_interval to 2000 ms? Right now this is disabled
by default.
The background is that I recently saw an incident where a blocking-lock
brownout escalated from a row-level problem to a complete system
outage, due to a combination of factors including a bug in golang's pgx
postgres client (PR 2481 has now been merged w a fix) and a pgbouncer
setup that was missing peers configuration. As a result, cancel
messages were getting dropped while postgres connections were waiting
on a blocked lock, golang aggresively timed out on context deadlines
and retried, and once the database reached max_connections the whole
system ground to a halt.
At the time I thought it was weird that postgres wasn't checking for
dead connections while those conns were waiting for locks; I spent a
bunch of time investigating this and reproduced it and wrote up what I
was able to figure out.
Then, yesterday, I saw a LinkedIn post from Marat at Data Egret who
mentioned that client_connection_check_interval exists.
Plugged this into my repro and confirmed it can prevent postgres from
escalating the blocking-lock brownout into a complete outage due to
connection exhaustion.
While a fix has been merged in pgx for the most direct root cause of
the incident I saw, this setting just seems like a good behavior to
make Postgres more robust in general. 2000 ms seemed like a fairly
safe/conservative starting point for discussion. Thoughts?
-Jeremy
PS. Some more details and graphs are at
https://ardentperf.com/2026/02/04/postgres-client_connection_check_interval/
--
To know the thoughts and deeds that have marked man's progress is to
feel the great heart throbs of humanity through the centuries; and if
one does not feel in these pulsations a heavenward striving, one must
indeed be deaf to the harmonies of life.
Helen Keller, The Story Of My Life, 1902, 1903, 1905, introduction by
Ralph Barton Perry (Garden City, NY: Doubleday & Company, 1954), p90.
| Attachment | Content-Type | Size |
|---|---|---|
| 0001-Change-default-client_connection_check_interval-to-2.patch | text/x-patch | 2.0 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Nikolay Samokhvalov | 2026-02-05 05:41:01 | Re: Add --system-identifier / -s option to pg_resetwal |
| Previous Message | Shlok Kyal | 2026-02-05 05:29:06 | Re: Skipping schema changes in publication |