Proposed patch for key managment

Attached is a patch for key management, which will eventually be part of
cluster file encryption (CFE), called TDE (Transparent Data Encryption)
by Oracle. It is an update of Masahiko Sawada's patch from July 31:

https://www.postgresql.org/message-id/CA+fd4k6RJwNvZTro3q2f5HSDd8HgyUc4CuY9U3e6Ran4C6TO4g@mail.gmail.com

Sawada-san did all the hard work, and I just redirected the patch. The
general outline of this CFE feature can be seen here:

https://wiki.postgresql.org/wiki/Transparent_Data_Encryption

The currently planned progression for this feature is to allow secure
retrieval of key encryption keys (KEK) outside of the database, then use
those to encrypt data keys that encrypt heap/index/tmpfile files.

This patch was changed from Masahiko Sawada's version by removing
references to non-cluster file encryption because having SQL-level keys
stored in this way was considered to have limited usefulness. I have
also remove references to SQL-level KEK rotation since that is best done
as a command-line too.

If most people approve of this general approach, and the design
decisions made, I would like to apply this in the next few weeks, but
this brings complications. The syntax added by this commit might not
provide a useful feature until PG 15, so how do we hide it from users.
I was thinking of not applying the doc changes (or commenting them out)
and commenting out the --help output.

Once this patch is applied, I would like to use the /dev/tty file
descriptor passing feature for the ssl_passphrase_command parameter, so
the ssl passphrase can be prompted from the terminal. (I am attaching
pass_fd.sh as a POC for how file descriptor handling works.) I would
also then write the KEK rotation command-line tool. After that, we can
start working on heap/index/tmpfile encryption using this patch as a
base.

Here is an example of the current patch in action, using a KEK like
'7CE7945EA2F7322536F103E4D7D91C21E52288089EF99D186B9A76D666EBCA66',
which is not echoed to the terminal:

$ initdb -R -c '/u/postgres/tmp/pass_fd.sh "Enter password:" %R'
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.
Cluster file encryption is enabled.

fixing permissions on existing directory /u/pgsql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... America/New_York
creating configuration files ... ok
running bootstrap script ...
--> Enter password:ok
performing post-bootstrap initialization ...
--> Enter password:ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

pg_ctl -D /u/pgsql/data -l logfile start

$ pg_ctl -R -l /u/pg/server.log start
waiting for server to start...
--> Enter password: done
server started

A non-matching passphrase looks like this:

$ pg_ctl -R -l /u/pg/server.log start
waiting for server to start...
--> Enter password: stopped waiting
pg_ctl: could not start server
Examine the log output.

$ tail -2 /u/pg/server.log
2020-12-02 16:32:34.045 EST [13056] FATAL: cluster passphrase does not match expected passphrase
2020-12-02 16:32:34.047 EST [13056] LOG: database system is shut down

--
Bruce Momjian <bruce(at)momjian(dot)us> https://momjian.us
EnterpriseDB https://enterprisedb.com

The usefulness of a cup is in its emptiness, Bruce Lee