shared-memory based stats collector

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: pgsql-hackers(at)postgresql(dot)org
Subject: shared-memory based stats collector
Date: 2018-06-29 08:34:18
Message-ID: 20180629.173418.190173462.horiguchi.kyotaro@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello.

This is intended to provide more stats like the following thread.

https://www.postgresql.org/message-id/20171010.192616.108347483.horiguchi.kyotaro@lab.ntt.co.jp

Most major obstracle for having more items in statistics
collector views comes from the mechanism to share the values
among backends. It is currently using a file. The stats collector
writes a file by triggers from backens then backens reads the
written file. Larger file makes the latency longer and we don't
have a spare bandwidth for additional statistics items.

Nowadays PostgreSQL has dynamic shared hash (dshash) so we can
use this as the main storage of statistics. We can share data
without a stress using this.

A PoC previously posted tried to use "locally copied" dshash but
it doesn't looks fine so I steered to different direction.

With this patch dshash can create a local copy based on dynhash.

This patch consists of tree files.

v1-0001-Give-dshash-ability-to-make-a-local-snapshot.patch

adds dshash to make a local copy backed by dynahash.

v1-0002-Change-stats-collector-to-an-axiliary-process.patch

change the stats collector to be a auxiliary process so that it
can attach dynamic shared memory.

v1-0003-dshash-based-stats-collector.patch

implements shared-memory based stats collector.

I'll put more detailed explanation later.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
v1-0001-Give-dshash-ability-to-make-a-local-snapshot.patch text/x-patch 7.2 KB
v1-0002-Change-stats-collector-to-an-axiliary-process.patch text/x-patch 10.2 KB
v1-0003-dshash-based-stats-collector.patch text/x-patch 80.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dilip Kumar 2018-06-29 09:03:20 Re: [WIP] [B-Tree] Retail IndexTuple deletion
Previous Message Yugo Nagata 2018-06-29 08:31:01 Re: Fix to not check included columns in ANALYZE on indexes