Re: [WIP] Allow pg_upgrade to copy segments of the same relfilenode in parallel

On Mon, Mar 21, 2022 at 05:34:31PM -0700, Andres Freund wrote:
> Hi,
>
> On 2022-02-01 21:57:00 -0500, Jaime Casanova wrote:
> > This patch adds a new option (-J num, --jobs-per-disk=num) in
> > pg_upgrade to speed up copy mode. This generates upto ${num}
> > processes per tablespace to copy segments of the same relfilenode
> > in parallel.
> >
> > This can help when you have many multi gigabyte tables (each segment
> > is 1GB by default) in different tablespaces (each tablespace in a
> > different disk) and multiple processors.
> >
> > In a customer's database (~20Tb) it went down from 6h to 4h 45min.
> >
> > It lacks documentation and I need help with WIN32 part of it, I created
> > this new mail to put the patch on the next commitfest.
>
> The patch currently fails on cfbot due to warnings, likely related due to the
> win32 issue: https://cirrus-ci.com/task/4566046517493760?logs=mingw_cross_warning#L388
>
> As it's a new patch submitted to the last CF, hasn't gotten any review yet and
> misses some platform support, it seems like there's no chance it can make it
> into 15?
>

Hi,

Because I have zero experience on the windows side of this, I will take
some time to complete that part.

Should we move this to the next commitfest (and make 16 the target for
this)?

--
Jaime Casanova
Director de Servicios Profesionales
SystemGuards - Consultores de PostgreSQL