Re: O(n) tasks cause lengthy startups and checkpoints

From: Nathan Bossart <nathandbossart(at)gmail(dot)com>
To: Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, "Bossart, Nathan" <bossartn(at)amazon(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Maxim Orlov <orlovmg(at)gmail(dot)com>, Amul Sul <sulamul(at)gmail(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: O(n) tasks cause lengthy startups and checkpoints
Date: 2022-07-02 22:05:54
Message-ID: 20220702220554.GA1044460@nathanxps13
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Jun 24, 2022 at 11:45:22AM +0100, Simon Riggs wrote:
> On Thu, 23 Jun 2022 at 18:15, Nathan Bossart <nathandbossart(at)gmail(dot)com> wrote:
>> I'm grateful for the discussion in this thread so far, but I'm not seeing a
>> clear path forward.
>
> +1 to add the new auxiliary process.

I went ahead and put together a new patch set for this in which I've
attempted to address most of the feedback from upthread. Notably, I've
abandoned 0007 and 0008, added a way for processes to request specific
tasks for the custodian, and removed all the checks for
ShutdownRequestPending.

I haven't addressed the existing transaction ID wraparound risk with the
logical replication files. My instinct is that this deserveѕ its own
thread, and it might need to be considered a prerequisite to this change
based on the prior discussion here.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Attachment Content-Type Size
v6-0001-Introduce-custodian.patch text/x-diff 19.9 KB
v6-0002-Also-remove-pgsql_tmp-directories-during-startup.patch text/x-diff 4.8 KB
v6-0003-Split-pgsql_tmp-cleanup-into-two-stages.patch text/x-diff 9.8 KB
v6-0004-Move-pgsql_tmp-file-removal-to-custodian-process.patch text/x-diff 5.7 KB
v6-0005-Move-removal-of-old-serialized-snapshots-to-custo.patch text/x-diff 4.9 KB
v6-0006-Move-removal-of-old-logical-rewrite-mapping-files.patch text/x-diff 7.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Nathan Bossart 2022-07-02 22:16:35 Re: replacing role-level NOINHERIT with a grant-level option
Previous Message Tom Lane 2022-07-02 21:31:54 Re: AIX support - alignment issues