Re: [HACKERS] More stats about skipped vacuums

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: sawada(dot)mshk(at)gmail(dot)com
Cc: robertmhaas(at)gmail(dot)com, michael(dot)paquier(at)gmail(dot)com, tgl(at)sss(dot)pgh(dot)pa(dot)us, pgsql-hackers(at)postgresql(dot)org
Subject: Re: [HACKERS] More stats about skipped vacuums
Date: 2018-02-06 10:24:37
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

At Tue, 6 Feb 2018 14:50:01 +0900, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote in <CAD21AoCRn6Q0wGG7UwGVsQJZbocNsRaZByJomUy+-GRkVH-i9A(at)mail(dot)gmail(dot)com>
> On Mon, Dec 11, 2017 at 8:15 PM, Kyotaro HORIGUCHI
> <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote:
> > I considered dshash for pgstat.c and the attached is a *PoC*
> > patch, which is not full-fledged and just working under a not so
> > concurrent situation.
> >
> > - Made stats collector an auxiliary process. A crash of stats
> > collector leads to a whole-server restarting.
> >
> > - dshash lacks capability of sequential scan so added it.
> >
> > - Also added snapshot function to dshash. It just copies
> > unerlying DSA segments into local memory but currently it
> > doesn't aquire dshash-level locks at all. I tried the same
> > thing with resize but it leads to very quick exhaustion of
> > LWLocks. An LWLock for the whole dshash would be required. (and
> > it is also useful to resize() and sequential scan.
> >
> > - The current dshash doesn't shrink at all. Such a feature also
> > will be required. (A server restart causes a shrink of hashes
> > in the same way as before but bloat dshash requires copying
> > more than necessary size of memory on takeing a snapshot.)
> >
> > The size of DSA is about 1MB at minimum. Copying entry-by-entry
> > into (non-ds) hash might be better than copying underlying DSA as
> > a whole, and DSA/DSHASH snapshot brings a kind of dirty..
> >
> >
> > Does anyone give me opinions or suggestions?
> >
> The implementation of autovacuum work-item has been changed (by commit
> 31ae1638) because dynamic shared memory is not portable enough. IIUC
> this patch is going to do the similar thing. Since stats collector is
> also a critical part of the server, should we consider whether we can
> change it? Or the portability problem is not relevant with this patch?

Thank you for the pointer. I digged out the following thread from
this and the the patch seems to be the consequence of the
discussion. I'll study it and think what to do on this.


Kyotaro Horiguchi
NTT Open Source Software Center

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro HORIGUCHI 2018-02-06 10:41:23 Re: Query running for very long time (server hanged) with parallel append
Previous Message Etsuro Fujita 2018-02-06 10:22:34 Incorrect grammar