Re: Really really slow select count(*)

From: Greg Smith <greg(at)2ndquadrant(dot)com>
To: Craig Ringer <craig(at)postnewspapers(dot)com(dot)au>
Cc: felix <crucialfelix(at)gmail(dot)com>, "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: Re: Really really slow select count(*)
Date: 2011-02-07 19:05:25
Message-ID: 4D504275.5040103@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Craig Ringer wrote:
> What would possibly help would be if Pg could fall back to lower
> shared_buffers automatically, screaming about it in the logs but still
> launching.

This is exactly what initdb does when it produces an initial setting for
shared_buffers that goes into the postgresql.conf file. It wouldn't be
hard to move that same logic into a loop that executed when startup
failed to allocated enough memory.

There are two problems here, one almost solved, the other more
philosphical. It used to be that max_fsm_pages and wal_buffers could be
large enough components to the allocation that reducing them might
actually be a necessary fix, too. With the removal of the former and a
method to automatically set the latter now available, the remaining
components to the shared memory sizing computation are probably possible
to try and fix automatically if the kernel limits are too low.

But it's unclear whether running in a degraded mode, where performance
might be terrible, with only a log message is preferrable to stopping
and forcing the DBA's attention toward the mistake that was made
immediately. Log files get rotated out, and it's not hard to imagine
this problem coming to haunt someone only a month or two later--by which
time the change to shared_buffers is long forgotten, and the log message
complaining about it lost too. Accordingly I would expect any serious
attempt to add some auto-reduction behavior to be beset with argument,
and I'd never consider writing such a thing as a result. Too many
non-controversial things I could work on instead.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Josh Berkus 2011-02-07 22:58:40 Re: Write-heavy pg_stats_collector on mostly idle server
Previous Message Scott Marlowe 2011-02-07 18:42:02 Re: Really really slow select count(*)