Re: autovacuum launcher process eating up 17G+ of ram?

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Nick <t32(at)2thebatcave(dot)com>
Cc: Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>, pgsql-admin(at)postgresql(dot)org, Alvaro Herrera <alvherre(at)commandprompt(dot)com>
Subject: Re: autovacuum launcher process eating up 17G+ of ram?
Date: 2010-10-25 19:57:10
Message-ID: 4232.1288036630@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Nick <t32(at)2thebatcave(dot)com> writes:
> It seems like the problem is a function of how many databases there are.

Yeah, I can reproduce this in HEAD. Create a hundred or so databases,
and make sure there's a stats table entry for each, eg

create database d0;
\c d0
create table t1 as select generate_series(1,100000) x;
create database d1;
\c d1
create table t1 as select generate_series(1,100000) x;
create database d2;
\c d2
create table t1 as select generate_series(1,100000) x;
...

Set autovacuum_naptime to 1sec, and watch it bloat. (It bloats faster
once it's run through the databases once and isn't mostly waiting for
children.)

It looks to me like the problem is that get_database_list()
intentionally pushes its result into a long-lived context, and then the
result is never cleaned up. It doesn't help any that it does a lot of
leaky things like heap_beginscan with CurrentMemoryContext totally
unnecessarily pointing at the long-lived context.

I'm not sure what the intent was here, but it's pretty broken as-is.
Alvaro, what did you have in mind? Should this be using a less
long-lived context, or do we need code to free an avw_dbase list?

regards, tom lane

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Gnanakumar 2010-10-26 05:32:06 Re: pg_ctl: server does not shut down
Previous Message Matt Chesler 2010-10-25 19:11:48 Problem with Streaming Replication/Hot Standby