| From: | Nikolay Samokhvalov <nik(at)postgres(dot)ai> |
|---|---|
| To: | Daniel Gustafsson <daniel(at)yesql(dot)se> |
| Cc: | Fabrizio Mello <fabrizio(at)planetscale(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, "mail(at)joeconway(dot)com" <mail(at)joeconway(dot)com> |
| Subject: | Re: Add contrib module pg_stat_log: cumulative statistics about server log messages |
| Date: | 2026-09-09 13:31:01 |
| Message-ID: | CAM527d9FX3FL7P63Ww9qVQfaqWoUDMMAp_GgSxWg+O2HWdigeQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Thu, Sep 3, 2026 at 00:58 Daniel Gustafsson <daniel(at)yesql(dot)se> wrote:
> > On 18 Aug 2026, at 13:23, Fabrizio Mello <fabrizio(at)planetscale(dot)com>
> wrote:
>
> > * Is contrib the right home, or would people rather see this as a
> > builtin stats kind with a core pg_stat_* view? Contrib felt right for
> > a first step since it exercises the custom stats API and is entirely
> > optional, but I'm happy to go either way.
>
> IMHO, contrib is not the right home for anything new. Either we think it's
> important for production and place it in core or it should live as an
> external
> extension.
>
> Your email didn't motivate why this extension should be in core instead of
> maintained as an external extension?
I strongly support having this in core, exposed through a pg_stat_* view,
rather than in contrib or an external extension.
Reason is quite straightforward: error counters / error analysis is
fundamental database monitoring.
For example, I want to alert when Postgres reports an internal error
(XX000), data corruption (XX001), or index corruption (XX002). Today,
standard statistics don't provide general counters by SQLSTATE. We have
rollbacks, deadlocks, recovery conflicts, and some other specific counters,
but they don't tell us whether these errors occurred.
Getting that information from logs is now the only way, but operationally
much harder than collecting counters:
- Logs mix diagnostic messages with statements and potentially sensitive
data / PII, such as parameter values. Giving a monitoring system access to
all of that is different from letting it read error counts. I personally
feel this when working with users of managed Postgres platforms. It's
usually not a problem to get pg_monitor access, and it's a completely
different story to get access to logs.
- Exporting pg_stat_* counters to Prometheus is straightforward and common.
Processing logs requires a separate pipeline, with additional permissions,
storage, resource usage, and failure modes.
• In managed Postgres, users may have SQL monitoring access but limited
control over log collection or which extensions are available.
- SQL path of analysis is also great for smaller clusters that cannot
justify full fledged monitoring stacks -- having ability to just
self-analyze and see if we had errors of some kinds is good enough for
smaller cases.
Counters would not replace logs. I want monitoring to tell me that Postgres
reported a corruption error, then use logs to investigate it. Collecting
the full diagnostic details should not be a prerequisite for detecting the
event, in my opinion.
An external extension is useful for developing and testing the
implementation, but I don't see it as the right long-term home for this
capability, especially considering how long it might take for it to
propagate to managed Postgres platforms.
Nik
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Sami Imseih | 2026-09-09 13:57:05 | Re: Improve WAIT FOR read-your-writes consistency doc |
| Previous Message | Sami Imseih | 2026-09-09 13:25:25 | Re: Reject WAIT FOR earlier in transaction-snapshot mode |