shared-memory based stats collector

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: pgsql-hackers(at)postgresql(dot)org
Subject: shared-memory based stats collector
Date: 2018-06-29 08:34:18
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


This is intended to provide more stats like the following thread.

Most major obstracle for having more items in statistics
collector views comes from the mechanism to share the values
among backends. It is currently using a file. The stats collector
writes a file by triggers from backens then backens reads the
written file. Larger file makes the latency longer and we don't
have a spare bandwidth for additional statistics items.

Nowadays PostgreSQL has dynamic shared hash (dshash) so we can
use this as the main storage of statistics. We can share data
without a stress using this.

A PoC previously posted tried to use "locally copied" dshash but
it doesn't looks fine so I steered to different direction.

With this patch dshash can create a local copy based on dynhash.

This patch consists of tree files.


adds dshash to make a local copy backed by dynahash.


change the stats collector to be a auxiliary process so that it
can attach dynamic shared memory.


implements shared-memory based stats collector.

I'll put more detailed explanation later.


Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
v1-0001-Give-dshash-ability-to-make-a-local-snapshot.patch text/x-patch 7.2 KB
v1-0002-Change-stats-collector-to-an-axiliary-process.patch text/x-patch 10.2 KB
v1-0003-dshash-based-stats-collector.patch text/x-patch 80.6 KB


Browse pgsql-hackers by date

  From Date Subject
Next Message Dilip Kumar 2018-06-29 09:03:20 Re: [WIP] [B-Tree] Retail IndexTuple deletion
Previous Message Yugo Nagata 2018-06-29 08:31:01 Re: Fix to not check included columns in ANALYZE on indexes