From: | "Bossart, Nathan" <bossartn(at)amazon(dot)com> |
---|---|
To: | Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> |
Cc: | "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: O(n) tasks cause lengthy startups and checkpoints |
Date: | 2021-12-02 21:31:17 |
Message-ID: | 5E58EBB2-3DAE-4400-9EBE-60D1FE4C5B10@amazon.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 12/1/21, 6:48 PM, "Bharath Rupireddy" <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
> +1 for the overall idea of making the checkpoint faster. In fact, we
> here at our team have been thinking about this problem for a while. If
> there are a lot of files that checkpoint has to loop over and remove,
> IMO, that task can be delegated to someone else (maybe a background
> worker called background cleaner or bg cleaner, of course, we can have
> a GUC to enable or disable it). The checkpoint can just write some
Right. IMO it isn't optimal to have critical things like startup and
checkpointing depend on somewhat-unrelated tasks. I understand the
desire to avoid adding additional processes, and maybe it is a bigger
hammer than what is necessary to reduce the impact, but it seemed like
a natural solution for this problem. That being said, I'm all for
exploring other ways to handle this.
> Another idea could be to parallelize the checkpoint i.e. IIUC, the
> tasks that checkpoint do in CheckPointGuts are independent and if we
> have some counters like (how many snapshot/mapping files that the
> server generated)
Could you elaborate on this? Is your idea that the checkpointer would
create worker processes like autovacuum does?
Nathan
From | Date | Subject | |
---|---|---|---|
Next Message | Bossart, Nathan | 2021-12-02 21:31:53 | Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file |
Previous Message | Tom Lane | 2021-12-02 21:22:21 | Dubious usage of TYPCATEGORY_STRING |