| From: | Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> |
|---|---|
| To: | andres(at)anarazel(dot)de |
| Cc: | sfrost(at)snowman(dot)net, alvherre(at)2ndquadrant(dot)com, michael(at)paquier(dot)xyz, thomas(dot)munro(at)gmail(dot)com, tomas(dot)vondra(at)2ndquadrant(dot)com, ah(at)cybertec(dot)at, magnus(at)hagander(dot)net, robertmhaas(at)gmail(dot)com, tgl(at)sss(dot)pgh(dot)pa(dot)us, pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: shared-memory based stats collector |
| Date: | 2020-10-06 01:06:44 |
| Message-ID: | 20201006.100644.1914636133592635832.horikyota.ntt@gmail.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Rebased on a previously committed WAL-stats patch.
I found a bug that the maximum interval was wrongly set to 600s
instead of 60s.
The previous version failed to flush local database stats for certain
condition. That behavior caused useless retries and finally a forced
flush that leads to contention. I fixed that and will measure
performance with this version.
Now that global stats are split into bgwriter stats and checkpointer
stats, that stats are updated only by one process each. However, they
are reset by client backends so LWLock is still needed to protect
them. To get rid of the LWLocks, pgstat_reset_shared_counters() is
changed so as to avoid scribble on the shared structs.
Finally archiver, bgwriter and checkpointer stats no longer need
LWLock to update, read and reset. Still reader-reader conflict on
StatsLock occurs but that doesn't affect writer processes.
WAL stats is written from many backends so it still requires LWLock to
reset, update and read.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
| Attachment | Content-Type | Size |
|---|---|---|
| v39-0001-sequential-scan-for-dshash.patch | text/x-patch | 8.8 KB |
| v39-0002-Add-conditional-lock-feature-to-dshash.patch | text/x-patch | 6.2 KB |
| v39-0003-Make-archiver-process-an-auxiliary-process.patch | text/x-patch | 17.7 KB |
| v39-0004-Shared-memory-based-stats-collector.patch | text/x-patch | 279.8 KB |
| v39-0005-Doc-part-of-shared-memory-based-stats-collector.patch | text/x-patch | 20.7 KB |
| v39-0006-Remove-the-GUC-stats_temp_directory.patch | text/x-patch | 13.6 KB |
| v39-0007-Exclude-pg_stat-directory-from-base-backup.patch | text/x-patch | 1.2 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Fujii Masao | 2020-10-06 01:54:27 | Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away |
| Previous Message | Tom Lane | 2020-10-06 00:42:15 | Recent failures on buildfarm member hornet |