Re: lots of updates on small table

On Fri, Jul 15, 2005 at 09:42:12AM +1000, Alison Winters wrote:

> > > Our application requires a number of processes to select and update rows
> > > from a very small (<10 rows) Postgres table on a regular and frequent
> > > basis. These processes often run for weeks at a time, but over the
> > > space of a few days we find that updates start getting painfully slow.
> > > We are running a full vacuum/analyze and reindex on the table every day,
> > Full vacuum, eh? I wonder if what you really need is very frequent
> > non-full vacuum. Say, once in 15 minutes (exact rate depending on dead
> > tuple rate.)
> >
> Is there a difference between vacuum and vacuum full?

Yes. Vacuum full is more aggresive in compacting the table. Though it
really works the same in the presence of long-running transactions:
tuples just can't be removed.

> The most recent output was this:
>
> INFO: --Relation public.plc_fldio--
> INFO: Pages 1221: Changed 3, reaped 256, Empty 0, New 0; Tup 108137: Vac 4176, Keep/VTL 108133/108133, UnUsed 19, MinLen 84, MaxLen 84; Re-using: Free/Avail. Space 445176/371836; EndEmpty/Avail. Pages 0/256.
> CPU 0.04s/0.14u sec elapsed 0.18 sec.
> INFO: Index plcpage_idx: Pages 315; Tuples 108137: Deleted 4176.
> CPU 0.03s/0.04u sec elapsed 0.14 sec.
> INFO: Rel plc_fldio: Pages: 1221 --> 1221; Tuple(s) moved: 0.
> CPU 0.03s/0.04u sec elapsed 0.36 sec.
> INFO: Analyzing public.plc_fldio

Hmm, so it seems your hourly vacuum is enough. I think the bloat theory
can be trashed. Unless I'm reading this output wrong; I don't remember
the details of this vacuum output.

> We'll up it to every 15 minutes, but i don't know if that'll help
> because even with the current vacuuming the updates are still getting
> slower and slower over the course of several days. What really puzzles
> me is why restarting the processes fixes it.

I wonder if the problem may be plan caching. I didn't pay full
attention to the description of your problem, so I don't remember if it
could be an issue, but it's something to consider.

> Does PostgreSQL keep some kind of backlog of transactions all for one
> database connection?

No. There could be a problem if you had very long transactions, but
apparently this isn't your problem.

> Isn't it normal to have processes that keep a single database
> connection open for days at a time?

I guess it depends on exactly what you do with it. I know of at least
one case where an app keeps a connection open for months, without a
problem. (It's been running for four or five years, and monthly
"uptime" for that particular daemon is not unheard of.)

--
Alvaro Herrera (<alvherre[a]alvh.no-ip.org>)
"Everybody understands Mickey Mouse. Few understand Hermann Hesse.
Hardly anybody understands Einstein. And nobody understands Emperor Norton."