Re: File content logging during execution of COPY queries (was: Better logging of COPY queries if log_statement='all')

From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Aleksander Alekseev <a(dot)alekseev(at)postgrespro(dot)ru>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Andrew Dunstan <andrew(at)dunslane(dot)net>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: File content logging during execution of COPY queries (was: Better logging of COPY queries if log_statement='all')
Date: 2016-10-20 12:50:04
Message-ID: 20161020125004.GL13284@tamriel.snowman.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Aleksander,

* Aleksander Alekseev (a(dot)alekseev(at)postgrespro(dot)ru) wrote:
> > The idea is to record application workload in real environment and write
> > a benchmark based on this record. Then using this benchmark we could try
> > different OS/DBMS configuration (or maybe hardware), find an extremum,
> > then change configuration in production environment.
> >
> > It's not always possible to change an application or even database (e.g.
> > to use triggers) for this purpose. For instance, if DBMS is provided as
> > a service.
> >
> > Currently PostgreSQL allows to record all workload _except_ COPY
> > queries. Considering how easily it could be done I think it's wrong.
> > Basically the only real question here is how it should look like in
> > postgresql.conf.
>
> OK, how about introducing a new boolean parameter named log_copy?
> Corresponding patch is attached.

The parameter would be better as 'log_copy_data', I believe. The actual
COPY command is already logged with just 'log_statement = all', of
course.

Also..

> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
> index 8c25b45..84a7542 100644
> --- a/doc/src/sgml/config.sgml
> +++ b/doc/src/sgml/config.sgml
> @@ -5205,6 +5205,20 @@ FROM pg_stat_activity;
> </listitem>
> </varlistentry>
>
> + <varlistentry id="guc-log-copy" xreflabel="log_copy">
> + <term><varname>log_copy</varname> (<type>boolean</type>)
> + <indexterm>
> + <primary><varname>log_copy</> configuration parameter</primary>
> + </indexterm>
> + </term>
> + <listitem>
> + <para>
> + Controls whether file content is logged during execution of
> + COPY queries. The default is <literal>off</>.
> + </para>
> + </listitem>
> + </varlistentry>

"file" isn't accurate here and I don't know that it actually makes sense
to log "COPY TO" data- we don't log the results of SELECT statements,
after all, and the use-case you outlined above (which I generally agree
is one we should consider) doesn't have any need for the data of "COPY
TO" statements to be in the log, it seems to me.

Can you elaborate on why we would want to log the data sent to the
client with a COPY TO command. If there is a reason, why wouldn't we
want to support that for SELECT and ... RETURNING commands too?

Thanks!

Stephen

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Joshua D. Drake 2016-10-20 13:34:42 Re: Disable autovacuum guc?
Previous Message Constantin S. Pan 2016-10-20 12:30:24 Re: Fun fact about autovacuum and orphan temp tables