Re: shared-memory based stats collector

From: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
To: alvherre(at)2ndquadrant(dot)com
Cc: andres(at)anarazel(dot)de, michael(at)paquier(dot)xyz, thomas(dot)munro(at)gmail(dot)com, tomas(dot)vondra(at)2ndquadrant(dot)com, ah(at)cybertec(dot)at, magnus(at)hagander(dot)net, robertmhaas(at)gmail(dot)com, tgl(at)sss(dot)pgh(dot)pa(dot)us, pgsql-hackers(at)postgresql(dot)org
Subject: Re: shared-memory based stats collector
Date: 2020-04-07 07:38:17
Message-ID: 20200407.163817.1988612531068457705.horikyota.ntt@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

At Fri, 03 Apr 2020 17:31:17 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> Conflicted with the commit 28cac71bd3 - SLRU stats.
>
> Rebased and fixed some issues.
> - A PoC (or a rush work) of refactoring of SLRU stats.
>
> I tried to combine it into the global stats hash, but the SLRU
> report functions are called within critical sections so memory
> allocation fails. The current pgstat module removes the local entry
> at successful flushing out to shared stats, so allocation at the
> first report is inevitable. In the attched version it is handled
> the same way with global stats. I continue seeking a way to
> combining it to the global stats hash.

I didn't find a way to consolidate into the general local stats
hash. The hash size could be large and the chance of allocation
failure is larger than other places where in-critical-section memory
allocation is allowed. Instead, in the attached, I separated
shared-SLRU lock from StatsLock and add the logic to avoid useless
scan on the array.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
v32-0001-Use-standard-crash-handler-in-archiver.patch text/x-patch 1.7 KB
v32-0002-sequential-scan-for-dshash.patch text/x-patch 8.4 KB
v32-0003-Add-conditional-lock-feature-to-dshash.patch text/x-patch 6.2 KB
v32-0004-Make-archiver-process-an-auxiliary-process.patch text/x-patch 17.6 KB
v32-0005-Shared-memory-based-stats-collector.patch text/x-patch 260.7 KB
v32-0006-Doc-part-of-shared-memory-based-stats-collector.patch text/x-patch 21.5 KB
v32-0007-Remove-the-GUC-stats_temp_directory.patch text/x-patch 9.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro Horiguchi 2020-04-07 07:48:01 Re: Don't try fetching future segment of a TLI.
Previous Message Kyotaro Horiguchi 2020-04-07 07:30:43 Re: [HACKERS] Restricting maximum keep segments by repslots