Re: shared-memory based stats collector - v69

From: Andres Freund <andres(at)anarazel(dot)de>
To: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
Cc: melanieplageman(at)gmail(dot)com, Justin Pryzby <pryzby(at)telsasoft(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: shared-memory based stats collector - v69
Date: 2022-04-05 03:05:06
Message-ID: 20220405030506.lfdhbu5zf4tzdpux@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Thanks for the reviews Justin, Thomas, David. I tried to incorporate the
feedback, with the exception of the ongoing discussion around
accessed_across_databases. I've also not renamed pg_stat_exists_stat() yet,
not clear who likes what :)

Changes in v69:

- merged feedback
- committed the first few commits, mostly pretty boring stuff
- added an architecture overview comment to the top of pgstat.c - not sure if
it makes sense to anybody but me (and perhaps Horiguchi-san)?
- merged "only reset pgstat data after crash recovery." into the main commit,
added tests verifying the behaviour of not resetting stats on a standby when
in SHUTDOWNED_IN_RECOVERY.
- drop variable-amount stats when loading on-disk file fails partway through,
I'd raised this earlier in [1]
- made most pgstat_report_stat() calls pass force = true. In worker.c, the
only possibly frequent caller, I instead added a pgstat_report_stat(true) to
the idle path.
- added a handful more tests, but mostly out of "test coverage vanity" ;)
- made the test output of 030_stats_cleanup_replica a bit more informative,
plus other minor cleanups

The one definite TODO I know of is
> - fix the bug around pgstat_report_stat() I wrote about at [3]
> [3] https://www.postgresql.org/message-id/20220402081648.kbapqdxi2rr3ha3w@alap3.anarazel.de

I'd hoped Horiguchi-san would chime in on that discussion...

Regards,

Andres

[1] https://www.postgresql.org/message-id/20220329191727.mzzwbl7udhpq7pmf%40alap3.anarazel.de

Attachment Content-Type Size
v69-0001-dsm-allow-use-in-single-user-mode.patch text/x-diff 2.4 KB
v69-0002-pgstat-stats-collector-references-in-comments.patch text/x-diff 33.2 KB
v69-0003-pgstat-add-pgstat_copy_relation_stats.patch text/x-diff 3.8 KB
v69-0004-pgstat-move-transactional-code-into-pgstat_xact..patch text/x-diff 10.8 KB
v69-0005-pgstat-introduce-PgStat_Kind-enum.patch text/x-diff 2.1 KB
v69-0006-pgstat-prepare-APIs-used-by-pgstatfuncs-for-shar.patch text/x-diff 14.9 KB
v69-0007-pgstat-scaffolding-for-transactional-stats-creat.patch text/x-diff 42.7 KB
v69-0008-pgstat-revise-replslot-API-in-preparation-for-sh.patch text/x-diff 4.7 KB
v69-0009-pgstat-rename-some-pgstat_send_-functions-to-pgs.patch text/x-diff 8.4 KB
v69-0010-pgstat-store-statistics-in-shared-memory.patch text/x-diff 369.8 KB
v69-0011-pgstat-add-pg_stat_force_next_flush.patch text/x-diff 3.7 KB
v69-0012-pgstat-utilize-pg_stat_force_next_flush-to-simpl.patch text/x-diff 22.6 KB
v69-0013-pgstat-add-pg_stat_exists_stat-for-easier-testin.patch text/x-diff 3.9 KB
v69-0014-pgstat-test-transaction-behaviour-2PC-function-s.patch text/x-diff 165.1 KB
v69-0015-pgstat-test-test-stats-interactions-with-streami.patch text/x-diff 7.3 KB
v69-0016-pgstat-test-stats-handling-of-restarts-including.patch text/x-diff 9.3 KB
v69-0017-pgstat-test-subscriber-stats-reset-and-drop.patch text/x-diff 11.4 KB
v69-0018-pgstat-test-resetting-of-stats.patch text/x-diff 13.2 KB
v69-0019-pgstat-test-recovery-conflicts.patch text/x-diff 11.2 KB
v69-0020-pgstat-test-extend-replication-slot-stat-tests.patch text/x-diff 11.6 KB
v69-0021-pgstat-remove-stats_temp_directory.patch text/x-diff 10.4 KB
v69-0022-pgstat-rename-STATS_COLLECTOR-GUC-group-to-STATS.patch text/x-diff 4.2 KB
v69-0023-pgstat-move-pgstat.c-to-utils-activity.patch text/x-diff 2.0 KB
v69-0024-pgstat-update-docs.patch text/x-diff 25.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Smith 2022-04-05 03:09:01 Re: Handle infinite recursion in logical replication setup
Previous Message Greg Stark 2022-04-05 03:04:04 Re: Temporary tables versus wraparound... again