Re: Background Processes and reporting

On Sat, Mar 12, 2016 at 12:22 AM, Andres Freund <andres(at)anarazel(dot)de> wrote:

> On 2016-03-11 23:53:15 +0300, Vladimir Borodin wrote:
> > It was many times stated in threads about waits monitoring [0, 1, 2]
> > and supported by different people, but ultimately waits information
> > was stored in PgBackendStatus.
>
> Only that it isn't. It's stored in PGPROC.

Yes. And this is good. Original concept of single byte PgBackendStatus of
Robert looks incompatible with any further development of wait monitoring.

> This criticism is true of
> the progress reporting patch, but a quick scan of the thread doesn't
> show authors of the wait events patch participating there.
>
>
> > Can’t we think one more time about implementation provided by Ildus
> > and Alexander here [3]?
>
> I don't think so. Afaics the proposed patch tried to do too many things
> at once, and it's authors didn't listen well to criticism.

Original patch really did too many things at once. But it was good as
prototype.
We should move forward by splitting it into many smaller parts. But we
didn't because disagreement about design.

Idea of individual time measurement of every wait event met criticism
because it might have high overhead [1]. This is really so at least for
Windows [2]. But accessing only current values wouldn't be very useful. We
anyway need to gather some statistics. Gathering it by sampling would be
both more expensive and less accurate for majority of systems. This is why
I proposed hooks to make possible platform dependent extensions. Robert
rejects hook because he is "not a big fan of hooks as a way of resolving
disagreements about the design" [3]. Besides that is actually not design
issues but platform issues...

Another question is wait parameters. We want to expose wait event with
some parameters. Robert rejects that because it *might* add additional
overhead [3]. When I proposed to fit something useful into hard-won
4-bytes, Roberts claims that it is "too clever" [4].

So, situation looks like dead-end. I have no idea how to convince Robert
about any kind of advanced functionality of wait monitoring to PostgreSQL.
I'm thinking about implementing sampling extension over current
infrastructure just to make community see that it sucks. Andres, it would
be very nice if you have any idea how to move this situation forward.

Another aspect is that EnterpriseDB offers waits monitoring in proprietary
fork [5].

> SQL Session/System wait tuning diagnostics
> The Dynamic Runtime Instrumentation Tools Architecture (DRITA) allows a
> DBA to query catalog views to determine the wait events that affect the
> performance of individual sessions or the system as a whole. DRITA records
> the *number of times each event occurs as well as the time spent waiting*;
> you can use this information to diagnose performance problems.

And they declare to measure time of individual wait events. This is
exactly thing which Robert resist so much. So, I suspect some kind of
hidden motivation here. Probably, EBD guys just don't want to loose
majority of their proprietary product over open source PostgreSQL.

1.
http://www.postgresql.org/message-id/CA+TgmobDC4fbFpa49J6oK-iKazYVhW71VSSoTpvmf4tdyzRoCQ@mail.gmail.com
2.
http://www.postgresql.org/message-id/CAPpHfdvFLevfEtzGjPrNreXfLiyrKSGkdu98Xo5Xx0hw5GuBNw@mail.gmail.com
3.
http://www.postgresql.org/message-id/CA+TgmoZ-8ZpoUM9BGtBUP1u4dUQhC-9EpEDLzyK0dG4pKMDUwQ@mail.gmail.com
4.
http://www.postgresql.org/message-id/CA+TgmobyF2VcHGhVoks7yRXa2WDE+wmMRMxYugHvx_W-0wXLyA@mail.gmail.com
5.
http://www.enterprisedb.com/products-services-training/products/postgres-plus-advanced-server?quicktabs_advanceservertab=3#quicktabs-advanceservertab

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company