Re: Parsing config files in a directory

Hi,

Greg Smith <gsmith(at)gregsmith(dot)com> writes:
> It sounds like there's a consensus brewing here on what should get done with
> this particular patch now. Let me try to summarize:
>
> -The new feature should be activated by allowing you to specify a directory
> to include in the postgresql.conf like this:
>
> includedir 'conf'

I parse the current status as always reading files in the
postgresql.conf.d directory located in the same place as the current
postgresql.conf file.

Tom had a reserve about allowing the user the control the overloading
behavior, but it appears that what we're trying to provide is a way for
tools not to fight against DBA but help him/her. So Greg Stark's idea do
sounds better: .d/ files are read first in alphabetical order,
then postgresql.conf is read. If the DBA want to manually edit the
configuration and be sure his edit will have effect, he just edits
postgresql.conf. No wondering.

> -Within that directory, only file names of the form "*.conf" will be
> processed. More flexibility is hard to implement and of questionable
> value.

The regexp is still to be agreed upon, [0-9a-zA-Z-_.]+.conf or sth.

> -The order they are processed in will be alphabetical. This allows (but
> doesn't explictly require) using the common convention of names like
> "99name" to get a really obvious ordering.

Yes.

> -The default postgresql.conf should be updated to end with the sample
> includedir statement shown above. This will make anything that goes into
> there be processed after the main file, and therefore override anything in
> it.

No, hardwired knowledge makes life easier without trading
capabilities. Loading is deterministic:
1. files in postgresql.conf.d in alphabetical order
2. postgresql.conf

This way the directory is for tools or common setup and the file for
local editing by the admin.

> -An intended purpose here is making tools easier to construct. It's
> impractical to expect every tool that touches files in the config directory
> to do an exhaustive sweep to find every other place there might be a
> conflict and comment them all out. The fact that pg_settings shows users
> the exact file and line they setting that is the active one is a good enough
> tool to allow DBAs to work through most of the problem cases.

If we want to insist on having both user comments and settings in the
files in postgresql.conf.d, I still think the best is to have there GUC
named file. First line contains current value. Rest of the file is
comments. Now you can even have SET PERSISTENT ... WITH COMMENT ... and
COMMENT ON GUC ...;

Then the pg_settings view could also embed the comments.

> And as far as how it impacts planning:
>
> -A future patch to initdb could move the changes it makes from the primary
> file to one in the config directory. It might make sense to use a name like
> 00initdb.conf to encourage a known good naming practice for files in the
> config directory; that doesn't need to get nailed down now though.

00-initdb.conf if you want some bikesheding to happen :)

> -This patch makes it easier to envision implementing a smaller default
> postgresql.conf, but it doesn't require such a change to be useful.

The postgresql.conf file could remain the same or not, and still is
intended for manual editing only.

> -SET PERSISTENT is still a bit away. This patch assists in providing a
> cleaner preferred way to implement that, and certainly doesn't make it
> harder to build.

If we stick to « dba is not supposed to manually edit any file in the
directory or things will get broken », then have either a
99-persistent.conf file or the one-file-per-GUC approach. The former
sounds easy to implement if we drop comments out of tool scope, the
latter is more flexible but looks ugly to most... (but you're forbidden
to have a look there).

> The issue of how to handle backing out changes that result
> in a non-functional server configuration is still there. And there's some
> support for the idea that the SQL interface should do more sanity checks to
> make sure its setting changes aren't being overridden by config files parsed
> later than we might expect from external tuning tools.

In the one-file-per-GUC / do-not-edit-any-postgresql.conf.d-file idea,
it's possible to have a convention for tools to manage history of settings.

Regards,
--
dim