Re: shared-memory based stats collector

From: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
To: alvherre(at)2ndquadrant(dot)com
Cc: andres(at)anarazel(dot)de, michael(at)paquier(dot)xyz, thomas(dot)munro(at)gmail(dot)com, tomas(dot)vondra(at)2ndquadrant(dot)com, ah(at)cybertec(dot)at, magnus(at)hagander(dot)net, robertmhaas(at)gmail(dot)com, tgl(at)sss(dot)pgh(dot)pa(dot)us, pgsql-hackers(at)postgresql(dot)org
Subject: Re: shared-memory based stats collector
Date: 2020-05-15 08:30:36
Message-ID: 20200515.173036.271465470355561419.horikyota.ntt@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

At Tue, 07 Apr 2020 16:38:17 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> At Fri, 03 Apr 2020 17:31:17 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> > Conflicted with the commit 28cac71bd3 - SLRU stats.
> >
> > Rebased and fixed some issues.
> > - A PoC (or a rush work) of refactoring of SLRU stats.
> >
> > I tried to combine it into the global stats hash, but the SLRU
> > report functions are called within critical sections so memory
> > allocation fails. The current pgstat module removes the local entry
> > at successful flushing out to shared stats, so allocation at the
> > first report is inevitable. In the attched version it is handled
> > the same way with global stats. I continue seeking a way to
> > combining it to the global stats hash.
>
> I didn't find a way to consolidate into the general local stats
> hash. The hash size could be large and the chance of allocation
> failure is larger than other places where in-critical-section memory
> allocation is allowed. Instead, in the attached, I separated
> shared-SLRU lock from StatsLock and add the logic to avoid useless
> scan on the array.

Maybe 29c3e2dd5a and recent doc change hit this. Rebased.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
v33-0001-Use-standard-crash-handler-in-archiver.patch text/x-patch 1.7 KB
v33-0002-sequential-scan-for-dshash.patch text/x-patch 8.4 KB
v33-0003-Add-conditional-lock-feature-to-dshash.patch text/x-patch 6.2 KB
v33-0004-Make-archiver-process-an-auxiliary-process.patch text/x-patch 17.6 KB
v33-0005-Shared-memory-based-stats-collector.patch text/x-patch 257.2 KB
v33-0006-Doc-part-of-shared-memory-based-stats-collector.patch text/x-patch 20.5 KB
v33-0007-Remove-the-GUC-stats_temp_directory.patch text/x-patch 9.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Atsushi Torikoshi 2020-05-15 08:47:41 Re: Is it useful to record whether plans are generic or custom?
Previous Message Kyotaro Horiguchi 2020-05-15 08:25:08 Re: pg_regress cleans up tablespace twice.