Re: logging in high performance systems.

On Mon, Jan 16, 2012 at 3:51 AM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> There is an important distinction to make here.  Statement logging is one of
> the largest producers of logging data you want to worry about optimizing for
> on a high performance system.  The decision about whether to log or not may
> need to be made by the hook.  Something that hooks EmitErrorReport has
> already lost an important opportunity to make a decision about whether the
> system is perhaps too busy to worry about logging right now at all; you've
> already paid a chunk of the logging overhead getting the log message to it.
>  I think both areas are going to be important to hook eventually.

I would dismissed this out of hand at this if you said it a year ago,
but I'm older and wiser now. At some point this cycle, I did some
benchmarking of the subtransaction abort path, since the slowness of
things like EXCEPTION blocks in PL/pgsql is a known sore point. I
don't remember the exact numbers anymore, but I do remember the
general picture, which is that constructing the error message is
shockingly expensive compared to anything else that we do in that
path. I dropped it at that point for lack of good ideas: it would be
awfully nice to postpone the error message construction until we know
that it's actually needed, but I don't see any clean (or even messy)
way of doing that.

I'm not sure if the effect is quite as significant for toplevel
transaction abort, because sending the message to the client is going
to cost something, so the relative cost of generating the error
message on a busy system will be less. But I still wouldn't like to
bet against it being significant if you're really hammering the
server.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company