Re: [PATCHES] serverlog rotation/functions

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Andreas Pflug <pgadmin(at)pse-consulting(dot)de>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Subject: Re: [PATCHES] serverlog rotation/functions
Date: 2004-07-15 22:39:53
Message-ID: 200407152239.i6FMdrM05282@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

Andreas Pflug wrote:
> > You do something that splits the value into directory name and file name
> > and removes every letter after %.
> >
> > /var/log
> > postgresql.log.%-%-%_%%%
> >
> > Another idea is to allow filename wildcards in the listing so it
> > becomes:
> >
> > SELECT *
> > FROM dir_listing('/var/log/postgresql.log.*-*-*_***') AS dir
> >
> > While that is nice, it doesn't match the functionality of opendir so we
> > are perhaps better with one that doesn't handle wildcards and we just do
> > the wildcard processing in the WHERE clause.
>
> Uh, this looks ugly.
>
> How about
> pg_logfile_list() RETURNS setof timestamp -- to list available logfiles
> pg_logfile_filename(timestamp) to return filename for that logfile

I don't see the need to return timestamps. If you select any empty
directory, you can just return the file names. The only reason you
might need a pattern is to distinguish pg log files from other log
files. If you want, create a server-side function that returns the file
name with the strftime() patterns converted to '*'.

> and generic
> pg_dir(wildcard_text)

Maybe pg_dir_ls().

OK, it would be nice if we could do a sed operation like:

sed 's/%./*/g'

but I don't know a way to do that without defining a function or pulling
in a procedural language, but if we could do it we could do:

pg_dir(echo log_destination | sed 's/%./*/g')

I think we will need a server-side function predefined to do this
conversion and we will call it:

pg_dir(glob_log_filename(log_destination))

> pg_file_length(filename_text)
> pg_file_read(filename_text, offs, len)
> pg_file_write(filename_text, contents_text)
> pg_file_delete(filename)

_delete should be _unlink.

> I'd like to have the logfile api straigt forward. I finally would like
> all server logging to go into a non-configurable DataDir subdirectory
> pg_log with filenames mangled by internal rules. Maybe it's a good idea
> to have the pid in the filename too, to detect postmaster restarts.

This will not work. What if there isn't sufficient room or
administrators want the log files somewher else? I don't see the value
in hardcoding the location. log_destination tells us enough.

> > Once we do this there will not be any backend writing to those files.
>
> Of course not, only the logging subprocess may write.
>
> > (We will need the log subprocess pid in shared memory so backends can
> > send signals to it.)
>
> Yes, because the inherited SysLoggerPID might become invalid in case the
> logger process crashes and is recreated.

OK.

> I am not sure how we will do file deletes but I
> > think each backend will have to do the delete itself rather than try to
> > pass the file name to the log process.
>
> Agreed.
>
> I think we will have to assume
> > the log file names increase in ordering so we know which one is the
> > current one. I can't think if a cleaner way to communicate this to the
> > backends except perhaps as you suggest as shared memory areas that
> > contains the name, but we will need a lock so the backends don't read it
> > while it is changing. That would be a nice feature.
>
> We can omit this, if we supply only a native function. In that case,
> it's up to the admin not to shoot himself into the foot.

Yes, we can improve this in 7.6 if we want.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Simon Riggs 2004-07-15 22:44:02 Re: Point in Time Recovery
Previous Message Devrim GUNDUZ 2004-07-15 22:18:23 Re: Point in Time Recovery

Browse pgsql-patches by date

  From Date Subject
Next Message Simon Riggs 2004-07-15 22:44:02 Re: Point in Time Recovery
Previous Message Peter Eisentraut 2004-07-15 21:20:46 Re: initdb authentication