Re: COPY-able csv log outputs

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Greg Smith <gsmith(at)gregsmith(dot)com>
Cc: pgsql-patches(at)postgresql(dot)org
Subject: Re: COPY-able csv log outputs
Date: 2007-05-30 19:50:42
Message-ID: 200705301950.l4UJogM02136@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-patches


Your patch has been added to the PostgreSQL unapplied patches list at:

http://momjian.postgresql.org/cgi-bin/pgpatches

It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.

---------------------------------------------------------------------------

Greg Smith wrote:
> The attached patch fixes all the issues I found in the original version of
> this code and completes the review I wanted to do. Someone else will need
> to take this from here. As I already mentioned, I can't comment on the
> quality of the piping implementation used to add this feature other than
> to say it worked for me.
>
> Here is a sample line generated by the current code that illustrates most
> of what I adjusted:
>
> 2007-05-28 13:45:14.028 EDT,"postgres","pgbench",465b1519.5f3d,laptop:32783,24381,17,idle,2007-05-28
> 13:44:57 EDT,60337,LOG,,"statement: select * from branches;"
>
> In addition to cleanup and bug fixes, notable changes made from the
> original version include:
>
> -Changed connection information to standard host:port syntax
>
> -Added a per process line number to allow a unique index. The code that
> supports that in the current logs is very local to log_line_prefix.
> Rather than try to share that information like is done for the timestamp,
> it seemed easier to just duplicate those few lines of code for the CSV
> output path. This could cause the CVS logs to have different values for
> the line number of a statement than the text format ones. I didn't feel
> that was a problem serious enough to justify the code refactoring that
> would be required to assure the line numbers were identical.
>
> -Added a new documentation section to the logging chapter devoted just to
> the csvlog feature. It gives a sample table and import syntax. I also
> gave recommendations on how to configure some related log file parameters
> that can interact badly with this feature. For example, I noticed that if
> log_rotation_size was set to a value, it could split the CSV lines in two;
> the result was two CVS files you couldn't import because of the partial
> lines in each. Since the rotation size feature causes other issues anyway
> that make importing more complicated, documenting the issue seemed
> sufficient.
>
> --
> * Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD
Content-Description:

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

In response to

Browse pgsql-patches by date

  From Date Subject
Next Message Tom Lane 2007-05-30 19:56:38 Re: Seq scans status update
Previous Message Neil Conway 2007-05-30 19:40:17 Re: boolean <=> text explicit casts