Re: Second attempt, roll your own autovacuum

From: Csaba Nagy <nagy(at)ecircle-ag(dot)com>
To: Christopher Browne <cbbrowne(at)acm(dot)org>
Cc: Postgres general mailing list <pgsql-general(at)postgresql(dot)org>
Subject: Re: Second attempt, roll your own autovacuum
Date: 2006-12-19 14:46:13
Message-ID: 1166539573.13028.251.camel@coppola.muc.ecircle.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

>From my POV, autovacuum is doing a very good job, with the exception of:

> - There might be time-based exclusions to the effect that large tables
> oughtn't be processed during certain periods (backup time?)

Either (per table!) exception or permission based control of when a
table can be vacuumed is needed to avoid vacuuming big tables during
peek business periods. While this can be alleviated by setting lower
vacuum cost settings, and it won't block anymore other vacuums, it will
still need the multiple vacuum stuff to still process small tables:

> - One might have *two* consumers, one that will only process small
> tables, so that those little, frequently updated tables can get
> handled quickly, and another consumer that does larger tables.
> Or perhaps that knows that it's fine, between 04:00 and 09:00 UTC,
> to have 6 consumers, and blow through a lot of larger tables
> simultaneously.

So one of the 2 might be enough. I guess time-based
exclusion/permissions are not that easy to implement, and also not easy
to set up properly... so what could work well is:

- allow a "priority" setting per table in pg_autovacuum;
- create a vacuum thread for each priority;
- each thread checks it's own tables to be processed based on the
priority setting from pg_autovacuum;
- there have to be a default priority for tables not explicitly set up
in pg_autovacuum;
- possibly set a per priority default vacuum cost and delay;

In 8.2 the different vacuum threads for the different priorities won't
step on each other toes, and the default settings for the priorities can
be used to create some easily manageable settings for vacuuming table
categories with different update/delete patterns.

There could be some preset priorities, but creating new ones would be
useful so the user can create one per table update/delete pattern.

Maybe priority is not the best word for this, but I can't think now on
other better...

Cheers,
Csaba.

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Angva 2006-12-19 15:01:41 Re: out of memory woes
Previous Message Jeff Amiel 2006-12-19 14:32:02 Re: MAGIC_MODULE and libc

Browse pgsql-hackers by date

  From Date Subject
Next Message Hans-Juergen Schoenig 2006-12-19 14:56:01 Re: Core dump in PL/pgSQL ...
Previous Message Andrew Dunstan 2006-12-19 14:45:15 Re: pg_restore fails with a custom backup file