Re: Re: Industrial-Strength Logging

On Sat, 03 Jun 2000 12:36:58 -0400 Tom Lane wrote:

> You mentioned the issue of trying to deal with out-of-disk-space errors
> for the log file, but there is another kind of resource exhaustion
> problem that should also be taken into account. Namely, inability to
> open the log file due to EMFILE (no kernel filetable slots left)
> errors.

s/EMFILE/ENFILE/, but yes.

> I doubt we can expect that syslogd has a good strategy for coping with
> this :-(.

if ((f->f_file = open(p, O_WRONLY|O_APPEND, 0)) < 0) {
f->f_type = F_UNUSED;
logerror(p);

'nuff said.

> If the daemon is of our own making, the first thought that
> comes to mind is to hold the previous logfile open until after we
> successfully open the new one. If we get a failure on opening the new
> file, we just keep logging into the old one, while periodically trying
> to rotate again.

Costs an extra file descriptor, but it's only one and it's temporary.
I can't see anything better to do.

> Seems like the next step should be for someone to take a close look at
> the several available log-daemon packages and see which of them looks
> like the best bet for our purposes. (I assume there's no good reason
> to roll our own from scratch...)

Suggestions anyone? I like Bernstein's tools, but they're not freely
distributable enough to be integrated into Postgres. The Apache
program would be an OK starting place ... if 100 lines of not quite
right code is better than a blank page.

The other tools I mentioned -- swatch and logsurfer -- are really
analysis programs, and while someone might use them for on-the-fly
data reduction they're not really what we want as a default. Anyone?

It wouldn't hurt to discuss the requirements a little bit more here,
too. There is a compile time option to postgres to add timestamps.
Should the log program do that instead? Should the log writing
program have any responsibility for filtering?

Bernstein's "multilog" can do this sort of thing, although I don't
expect the interface to be to many peoples' liking:

http://cr.yp.to/daemontools/multilog.html

Regards,

Giles

P.S. I sent two copies of the message that Tom was replying to to
-hackers; I saw none come back. Very peculiar, since the one was sent
from an address subscribed to pgsql-loophole and one from an address
actually subscribed to the list.