Skip site navigation (1) Skip section navigation (2)

Re: Deleted WAL files held open by backends in Linux

From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Deleted WAL files held open by backends in Linux
Date: 2009-12-01 15:03:32
Message-ID: 4B14DBE4020000250002CE6E@gw.wicourts.gov (view raw or flat)
Thread:
Lists: pgsql-hackers
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
 
> How old were the sessions you were looking at?
 
Days to months old.
 
> If we think this is worth doing something about
> (I'm not convinced yet)
 
Once one knows about the issue, it's only a minor annoyance, and
that infrequently, so it's not worth anything that would take
significant effort or would cause any measurable performance hit.
 
> then the answer would be to forcibly close a backend's open WAL
> file in some reasonably seldom-used code path.  One possibility
> that comes to mind is to do it in ProcessCatchupEvent(), which
> will be invoked in approximately the right circumstances: a
> backend that is sitting idle for a long time within an otherwise
> busy system.  That wouldn't be a 100% solution, because if the
> backend is handling a steady stream of queries it will likely
> never run ProcessCatchupEvent().  But the places that would be
> 100% (like transaction commit) would probably entail too much of a
> performance hit from repeatedly opening and closing WAL files.
 
In our case there would tend to be bursts of activity on some of
these connections, and they would usually have a dribble of activity
from the monitoring system.  This dribble would consist of a couple
sub-millisecond SELECT statements once every five or ten seconds. 
(I have no clue whether that's frequent enough to suppress the
ProcessCatchupEvent() call.)
 
Is there a reasonably cheap way to check whether the backend has a
WAL file open and whether that one is the current append target?  As
you point out, the need to actually close such a file would be
infrequent, and it seems as though when the need exists, it wouldn't
be a matter of *whether* it would need to be closed, just *when*. 
Currently it sounds like we would often wind up doing it the next
time we try to run a query which has to flush dirty buffers, or when
the connection is being closed.  It seems like those might not be
the ideal time to add the overhead of the close, so we would just
need to find someplace which isn't worse.
 
-Kevin

In response to

Responses

pgsql-hackers by date

Next:From: Andrew DunstanDate: 2009-12-01 15:04:30
Subject: Re: CommitFest status/management
Previous:From: Florian WeimerDate: 2009-12-01 15:01:19
Subject: Re: Block-level CRC checks

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group