make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

From: Masahiro Ikeda <ikedamsh(at)oss(dot)nttdata(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.
Date: 2021-03-15 23:50:25
Message-ID: f9a3654d70c2e73da4efbdb0925b2739@oss.nttdata.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

This thread came from another thread about collecting the WAL
stats([1]).

Is it better to make the stats collector shutdown without writing the
stats files
if the immediate shutdown is requested?

There was a related discussion([2]) although it's stopped on 12/1/2016.
IIUC, the thread's consensus are

(1) It's useful to make the stats collector shutdown quickly without
writing the stats files
if the immediate shutdown is requested in some cases because there
is a possibility
that it takes a long time until the failover happens. The possible
reasons are that
disk write speed is slow, the stats files are big, and so on. And
there is no negative impact
on the users because all stats files are removed in a crash recovery
phase now.

(2) As another aspect, it needs to change the behavior removing all
stats files because autovacuum
uses the stats. There are some ideas, for example writing the stats
files every X minutes
(using wal or another mechanism) and even if a crash occurs, the
startup process can restore
the stats with slightly low freshness. But, it needs to find a way
how to handle the stats files
when deleting on PITR rewind or stats collector crash happens.

I agree that the above consensus and I think we can make separate two
patches.
The first one is for (1) and the second one is for (2).

Why don't you apply the patch for (1) first?
I attached the patch based on tsunakawa-san's patch([2]).
(v1-0001-pgstat_avoid_writing_on_sigquit.patch)

At this time, there are no cons points for the users because
the stats files are removed in a crash recovery phase as pointed in the
discussion.

[1]
https://www.postgresql.org/message-id/c616cf24bf4ecd818f7cab0900a40842%40oss.nttdata.com
[2]
https://www.postgresql.org/message-id/flat/0A3221C70F24FB45833433255569204D1F5EF25A%40G01JPEXMBYT05

Regards,
--
Masahiro Ikeda
NTT DATA CORPORATION

Attachment Content-Type Size
v1-0001-pgstat_avoid_writing_on_sigquit.patch text/x-diff 3.2 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Geoghegan 2021-03-16 00:00:33 Re: New IndexAM API controlling index vacuum strategies
Previous Message Andreas Karlsson 2021-03-15 23:27:25 Re: [HACKERS] GSoC 2017: Foreign Key Arrays