Re: O(n) tasks cause lengthy startups and checkpoints

From:	Nathan Bossart <nathandbossart(at)gmail(dot)com>
To:	"Bossart, Nathan" <bossartn(at)amazon(dot)com>
Cc:	Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Maxim Orlov <orlovmg(at)gmail(dot)com>, Amul Sul <sulamul(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Bruce Momjian <bruce(at)momjian(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject:	Re: O(n) tasks cause lengthy startups and checkpoints
Date:	2022-02-17 00:50:57
Message-ID:	20220217005057.GA3201038@nathanxps13
Views:	Whole Thread | Raw Message | Download mbox | Resend email
Thread:	2021-12-01 20:24:25 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>  2021-12-01 21:35:00 from SATYANARAYANA NARLAPURAM <satyanarlapuram(at)gmail(dot)com>  2021-12-01 22:56:32 from Andres Freund <andres(at)anarazel(dot)de>   2021-12-02 00:19:19 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>    2021-12-02 02:05:03 from "Euler Taveira" <euler(at)eulerto(dot)com>     2021-12-02 21:19:16 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>  2021-12-02 02:47:06 from Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>   2021-12-02 21:31:17 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>    2021-12-03 13:56:46 from Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>     2021-12-03 18:20:44 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>      2021-12-06 11:43:44 from Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> 📎       2021-12-06 19:22:03 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>        2021-12-10 19:03:17 from "Bossart, Nathan" <bossartn(at)amazon(dot)com> 📎         2021-12-13 13:53:37 from Robert Haas <robertmhaas(at)gmail(dot)com>          2021-12-13 17:19:35 from Justin Pryzby <pryzby(at)telsasoft(dot)com>           2021-12-13 18:30:43 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>          2021-12-13 18:21:02 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>           2021-12-13 20:36:24 from Robert Haas <robertmhaas(at)gmail(dot)com>            2021-12-13 23:05:46 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>             2021-12-14 16:59:50 from Bruce Momjian <bruce(at)momjian(dot)us>              2021-12-14 20:09:59 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>               2021-12-14 20:13:38 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>                2021-12-14 20:23:57 from "Bossart, Nathan" <bossartn(at)amazon(dot)com> 📎                 2022-01-02 21:26:01 from Andres Freund <andres(at)anarazel(dot)de>                  2022-01-03 06:59:04 from Amul Sul <sulamul(at)gmail(dot)com>                   2022-01-05 21:28:28 from "Bossart, Nathan" <bossartn(at)amazon(dot)com> 📎                    2022-01-14 11:41:46 from Maxim Orlov <orlovmg(at)gmail(dot)com>                     2022-01-14 19:16:01 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>                      2022-01-15 07:25:43 from Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>                       2022-01-18 20:00:41 from "Bossart, Nathan" <bossartn(at)amazon(dot)com>                        2022-02-11 18:02:49 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                         2022-02-17 00:50:57 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                          2022-02-17 01:50:52 from Andres Freund <andres(at)anarazel(dot)de>                           2022-02-17 04:14:04 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                            2022-02-17 06:59:38 from Andres Freund <andres(at)anarazel(dot)de>                             2022-02-17 18:23:37 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                              2022-02-17 19:27:09 from Andres Freund <andres(at)anarazel(dot)de>                               2022-02-17 21:00:22 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                2022-02-17 22:28:29 from Andres Freund <andres(at)anarazel(dot)de>                                 2022-02-17 22:58:38 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                  2022-02-17 23:12:47 from Andres Freund <andres(at)anarazel(dot)de>                                   2022-02-18 16:44:54 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                    2022-02-18 20:51:11 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                     2022-03-17 23:07:58 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                     2022-06-23 11:58:15 from Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>                                      2022-06-23 13:46:28 from Robert Haas <robertmhaas(at)gmail(dot)com>                                       2022-06-23 17:13:26 from Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>                                       2022-06-23 17:15:52 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                        2022-06-24 10:45:22 from Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>                                         2022-07-02 22:05:54 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                          2022-07-02 22:54:56 from Andres Freund <andres(at)anarazel(dot)de>                                           2022-07-03 17:07:54 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                            2022-07-03 17:27:32 from Andres Freund <andres(at)anarazel(dot)de>                                             2022-07-06 16:51:10 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                              2022-08-11 23:09:21 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                               2022-08-24 16:46:24 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                2022-09-02 22:07:44 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                 2022-09-23 17:41:54 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                  2022-11-06 22:38:42 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                   2022-11-24 00:19:07 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                    2022-11-24 17:31:02 from Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>                                                     2022-11-27 23:34:34 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                      2022-11-28 13:08:57 from Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>                                                       2022-11-28 18:31:29 from Andres Freund <andres(at)anarazel(dot)de>                                                        2022-11-28 18:37:01 from Robert Haas <robertmhaas(at)gmail(dot)com>                                                         2022-11-28 23:40:39 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                          2022-11-29 12:02:44 from Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>                                                           2022-11-30 03:56:53 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                            2022-11-30 05:18:33 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                             2022-11-30 11:22:44 from Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>                                                              2022-11-30 11:57:10 from Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>                                                               2022-12-01 21:40:26 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                                2022-12-02 06:41:35 from Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>                                                                 2022-12-02 19:15:07 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                                                  2022-12-06 07:28:28 from Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>                                                                   2023-02-03 05:48:08 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                                    2023-02-17 23:43:44 from Nathan Bossart <nathandbossart(at)gmail(dot)com> 📎                                                                     2023-04-02 17:40:05 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>                                                                      2023-04-02 18:42:26 from Andres Freund <andres(at)anarazel(dot)de>                                                                       2023-04-02 19:50:05 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                                                        2023-04-02 20:37:38 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>                                                                         2023-04-02 21:31:52 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                                                          2023-04-04 03:36:26 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                                                           2023-07-04 07:30:43 from Daniel Gustafsson <daniel(at)yesql(dot)se>                                                                            2023-07-04 18:37:36 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                                                      2023-04-02 19:30:30 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                                                       2023-04-02 20:23:05 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>                                                                        2023-04-02 21:17:42 from Nathan Bossart <nathandbossart(at)gmail(dot)com>                                                            2022-11-30 06:22:49 from Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>
Lists:	pgsql-hackers

Attachment	Content-Type	Size
v5-0001-Introduce-custodian.patch	text/x-diff	18.6 KB
v5-0002-Also-remove-pgsql_tmp-directories-during-startup.patch	text/x-diff	4.8 KB
v5-0003-Split-pgsql_tmp-cleanup-into-two-stages.patch	text/x-diff	9.8 KB
v5-0004-Move-pgsql_tmp-file-removal-to-custodian-process.patch	text/x-diff	6.4 KB
v5-0005-Move-removal-of-old-serialized-snapshots-to-custo.patch	text/x-diff	4.3 KB
v5-0006-Move-removal-of-old-logical-rewrite-mapping-files.patch	text/x-diff	8.1 KB
v5-0007-Use-syncfs-in-CheckPointLogicalRewriteHeap-for-sh.patch	text/x-diff	3.2 KB
v5-0008-Move-removal-of-spilled-logical-slot-data-to-cust.patch	text/x-diff	14.3 KB

In response to

• Re: O(n) tasks cause lengthy startups and checkpoints at 2022-02-11 18:02:49 from Nathan Bossart

Responses

• Re: O(n) tasks cause lengthy startups and checkpoints at 2022-02-17 01:50:52 from Andres Freund

Browse pgsql-hackers by date

	From	Date	Subject
Next Message	Nathan Bossart	2022-02-17 01:10:53	Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file
Previous Message	Michael Paquier	2022-02-17 00:48:12	Re: pgsql: Add TAP test to automate the equivalent of check_guc