Re: Flush some statistics within running transactions

From: Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com>
To: Sami Imseih <samimseih(at)gmail(dot)com>
Cc: Michael Paquier <michael(at)paquier(dot)xyz>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com>
Subject: Re: Flush some statistics within running transactions
Date: 2026-02-23 08:14:45
Message-ID: aZwMdVBYobV41duU@ip-10-97-1-34.eu-west-3.compute.internal
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On Sun, Feb 22, 2026 at 08:12:06PM -0600, Sami Imseih wrote:
> > > I took a look at this today out of interest,
> >
> > Thanks!
> >
> > > so, instead of calling IS_INJECTION_POINT_ATTACHED macro which is
> >
> > I think that not calling IS_INJECTION_POINT_ATTACHED() but only relying on
> > "ifdef USE_INJECTION_POINTS" would set the tiny timeout value to the entire test
> > suit.
>
> Yes, you are right about that.

And even if we are able to set a tiny timeout to say N in the test, the test
would still need a pg_sleep(N+<something>). So I wonder if the best way is not to
re-introduce pg_stat_force_anytime_flush() that was in v6?

> To Michael's earlier comments:
>
> For variable-length statistics, perhaps we can do things a bit
> differently than what is currently proposed. 0005 requires
> a relation anytime stat update to call
> pgstat_schedule_anytime_update(). This is done this way because
> it allows long-running queries to update their stats every
> stats_flush_interval using a timeout.
>
> But maybe what we should be doing for variable-numbered stats is
> to schedule an anytime update whenever a "transaction goes idle".

I think the logic for fixed stats and variable stats should be the same. If
not we could observe discrepancies: for example a long running select could
genereate reads/hits IO visible in pg_stat_io but tuples_returned, tuples_fetched,
blocks_fetched or blocks_hit would not be updated until the session goes idle.

> I find it odd to ask applications/clients to trigger a flush. I am not saying
> that we should not offer such an API, especially if someone want to flush
> stats more frequently than stats_flush_interval,

Yeah, and that could help with the tests to avoid the sleep.

> but there should be
> the ability for core to handle this automatically outside of the transaction
> boundaries.

Agreed. I think that public facing API could be an addition, not a replacement.

> One comment about the current test. I think there is a bug that was
> missed in the earlier review. For the var_anytime_update, we need to
> have an escape before the pipe.
>
> -like($result, qr/^entry2|2|/m,
> +like($result, qr/^entry2\|2\|/m,

Oh right, thanks! The attach rewrites it like the fixed test instead.

Also the attached introduces a mandatory rebase due to 308622edf17.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachment Content-Type Size
v11-0001-Add-pgstat_report_anytime_stat-for-periodic-stat.patch text/x-diff 42.9 KB
v11-0002-Add-anytime-flush-tests-for-custom-stats.patch text/x-diff 9.1 KB
v11-0003-Add-GUC-to-specify-non-transactional-statistics-.patch text/x-diff 10.0 KB
v11-0004-Remove-useless-calls-to-flush-some-stats.patch text/x-diff 7.7 KB
v11-0005-Change-RELATION-and-DATABASE-stats-to-anytime-fl.patch text/x-diff 34.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2026-02-23 08:21:15 Optimize SELECT * in EXISTS
Previous Message jian he 2026-02-23 08:04:55 Re: Non-text mode for pg_dumpall