Re: idea: log_statement_sample_rate - bottom limit for sampling

From: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>, Adrien Nayrat <adrien(dot)nayrat(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: idea: log_statement_sample_rate - bottom limit for sampling
Date: 2019-07-30 21:17:59
Message-ID: 20190730211759.zqzt7xfchvmuja4i@development
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Tue, Jul 30, 2019 at 03:43:58PM -0400, Tom Lane wrote:
>Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> writes:
>> I've started reviewing this patch, thinking that maybe I could get it
>> committed as it's marked as RFC. In general I agree with having this
>> fuature, but I think we need to rethink the GUC because the current
>> approach is just confusing.
>> ...
>> What I think we should do instead is to use two minimum thresholds.
>> 1) log_min_duration_sample - enables sampling of commands, using the
>> existing GUC log_statement_sample_rate
>> 2) log_min_duration_statement - logs all commands exceeding this
>> I think this is going to be much easier for users to understand.
>I agree with Tomas' idea.
>> The one difference between those approaches is in how they work with
>> existing current settings. That is, let's say you have
>> log_min_duration_statement = 1000
>> log_statement_sample_rate = 0.01
>> then no queries below 1000ms will be logged, and 1% of longer queries
>> will be sampled. And with the original config (as proposed in v3 of the
>> patch), this would still work the same way.
>> With the new approach (two min thresholds) it'd behave differently,
>> because we'd log *all* queries longer than 1000ms (not just 1%). And
>> whether we'd sample any queries (using log_statement_sample_rate) would
>> depend on how we'd pick the default value for the other threshold.
>Well, we do not need to have a backwards-compatibility problem
>here, because we have yet to release a version containing
>log_statement_sample_rate. I do not think it's too late to decide
>that v12's semantics for that are broken, and either revert that
>patch in v12, or back-patch a fix to make it match this idea.

I'm willing to try fixing this to salvage the feature for v12. The
question is how would that fix look like - IMO we'd need to introduce
the new threshold GUC, essentially implementing what this thread is
about. It's not a complex patch, but it kinda flies in the face of
feature freeze. OTOH if we call it a fix ...

The patch itself is not that complicated - attached is a WIP version,
(particularly) the docs need more work.


Tomas Vondra
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment Content-Type Size
log-sample-rework.patch text/plain 8.2 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message 毛瑞嘉 2019-07-30 21:43:05 [Patch] Adding CORRESPONDING/CORRESPONDING BY to set operation
Previous Message Ashwin Agrawal 2019-07-30 20:54:59 Re: heapam_index_build_range_scan's anyvisible