Re: shared-memory based stats collector

From: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
To: alvherre(at)2ndquadrant(dot)com
Cc: andres(at)anarazel(dot)de, michael(at)paquier(dot)xyz, thomas(dot)munro(at)gmail(dot)com, tomas(dot)vondra(at)2ndquadrant(dot)com, ah(at)cybertec(dot)at, magnus(at)hagander(dot)net, robertmhaas(at)gmail(dot)com, tgl(at)sss(dot)pgh(dot)pa(dot)us, pgsql-hackers(at)postgresql(dot)org
Subject: Re: shared-memory based stats collector
Date: 2020-04-03 08:31:17
Message-ID: 20200403.173117.818436775609909589.horikyota.ntt@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Conflicted with the commit 28cac71bd3 - SLRU stats.

Rebased and fixed some issues.

At Wed, 01 Apr 2020 17:37:23 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> > Conflicted with 616ae3d2b0, so rebased.
>
> I made some cleanup. (v30)
>
> - Added comments for members of dshash_seq_scans.
> - Some style fix and comment fix of dshash.
>
> - Cleaned up more usage of the word "stat(istics) collector" in comments,
> - Changed the GUC attribute STATS_COLLECTOR to STATS_ACTIVITY
> - Removed duplicate setup of MyBackendType and ps display in PgArchiverMain
> - Removed B_STATS_COLLECTOR from BackendType and removed related code.
> - Corrected the comment of PgArchiverMain, which mentioned "argv/argc".

I made further cleanups.

- Removed wrongly added BACKEND_TYPE_ARCHIVER.

- Moved archiver latch from XLogCtlData to ProcGlobal.

- Removed XLogArchiverStart/End/Wakeup from xlog.h, where anyway was
not the proper place.

- pgarch_MainLoop start the loop with wakened = true when both
notified or timed out. Otherwise time_to_stop is set and exits from
the loop immediately. So the variable wakened is actually
useless. Removed it.

- A PoC (or a rush work) of refactoring of SLRU stats.

I tried to combine it into the global stats hash, but the SLRU
report functions are called within critical sections so memory
allocation fails. The current pgstat module removes the local entry
at successful flushing out to shared stats, so allocation at the
first report is inevitable. In the attched version it is handled
the same way with global stats. I continue seeking a way to
combining it to the global stats hash.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
v31-0001-Use-standard-crash-handler-in-archiver.patch text/x-patch 1.7 KB
v31-0002-sequential-scan-for-dshash.patch text/x-patch 8.4 KB
v31-0003-Add-conditional-lock-feature-to-dshash.patch text/x-patch 6.2 KB
v31-0004-Make-archiver-process-an-auxiliary-process.patch text/x-patch 17.6 KB
v31-0005-Shared-memory-based-stats-collector.patch text/x-patch 259.8 KB
v31-0006-Doc-part-of-shared-memory-based-stats-collector.patch text/x-patch 21.5 KB
v31-0007-Remove-the-GUC-stats_temp_directory.patch text/x-patch 9.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Pavel Stehule 2020-04-03 08:38:50 Re: [Proposal] Global temporary tables
Previous Message Amit Kapila 2020-04-03 08:26:51 Re: User Interface for WAL usage data