Re: Parallel Inserts in CREATE TABLE AS

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
Cc: "Tang, Haiying" <tanghy(dot)fnst(at)cn(dot)fujitsu(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Zhihong Yu <zyu(at)yugabyte(dot)com>, Luc Vlaming <luc(at)swarm64(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, "Hou, Zhijie" <houzj(dot)fnst(at)cn(dot)fujitsu(dot)com>
Subject: Re: Parallel Inserts in CREATE TABLE AS
Date: 2021-05-27 04:13:07
Message-ID: CAA4eK1+M0ymNbJ6=jorF6tziv6J4XnC90muP3=kBeiz2NpNnXw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, May 26, 2021 at 5:51 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, May 26, 2021 at 5:28 PM Bharath Rupireddy
> <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
> >
> > On Fri, May 21, 2021 at 3:46 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > >
> > > On Fri, Mar 19, 2021 at 11:02 AM Bharath Rupireddy
> > > <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
> > > >
> >
> > > The other possibility could
> > > be that the free pages added to FSM by one worker are not being used
> > > by another worker due to some reason. Can we debug and check if the
> > > pages added by one worker are being used by another worker?
> >
> > I tried to explain it at [1]. Please have a look.
> >
>
> I have read it but I think we should try to ensure practically what is
> happening because it is possible that first time worker checked in FSM
> without taking relation extension lock, it didn't find any free page,
> and then when it tried to acquire the conditional lock, it got the
> same and just extended the relation by one block. So, in such a case
> it won't be able to use the newly added pages by another worker. I am
> not sure any such thing is happening here but I think it is better to
> verify it in some way. Also, I am not sure if just getting the info
> about the relation extension lock is sufficient?
>

One idea to find this out could be that we have three counters for
each worker which counts the number of times each worker extended the
relation in bulk, the number of times each worker extended the
relation by one block, the number of times each worker gets the page
from FSM. It might be possible that with this we will be able to
figure out why there is a difference between your and Hou-San's
results.

--
With Regards,
Amit Kapila.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2021-05-27 04:16:56 Re: Speed up pg_checksums in cases where checksum already set
Previous Message Tom Lane 2021-05-27 04:11:38 Re: Move pg_attribute.attcompression to earlier in struct for reduced size?