Re: Parallel INSERT (INTO ... SELECT ...)

From: Greg Nancarrow <gregn4422(at)gmail(dot)com>
To: "Hou, Zhijie" <houzj(dot)fnst(at)cn(dot)fujitsu(dot)com>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Antonin Houska <ah(at)cybertec(dot)at>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Parallel INSERT (INTO ... SELECT ...)
Date: 2021-01-22 01:42:15
Message-ID: CAJcOf-dSjTehL1X+Z0RwWe9MgQLhipSnuKEZ5BnQbOExnfi2Ow@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Jan 22, 2021 at 12:08 PM Hou, Zhijie <houzj(dot)fnst(at)cn(dot)fujitsu(dot)com> wrote:
>
> >
> > I think that's a good idea, so I'll make that update in the next version
> > of the patch.
> > I do notice, however, that there seems to be quite a few places in the Postgres
> > code where RelationGetIndexList() is being called without a corresponding
> > list_free() being called - obviously instead relying on it being deallocated
> > when the caller's memory-context is destroyed.
> >
>
> Yes, it will be deallocated when the caller's memory-context is destroyed.
>
> Currently, parallel safety-check check each partition.
> I am just a little worried about if there are lots of partition here, it may cause high memory use.
>
> And there is another place like this:
>
> 1.
> + conbin = TextDatumGetCString(val);
> + check_expr = stringToNode(conbin);
>
> It seems we can free the cobin when not used(for the same reason above).
> What do you think ?
>
>

Yes, I think you're right, we should pfree conbin after converting to
Node, to minimize memory usage.
Again, it's interesting that existing Postgres code, when looping
through all of the constraints, doesn't do this.
Hmmm. I'm wondering if there is a performance reason behind this -
avoiding multiple calls to pfree() and just relying on it to be
deallocated in one hit, when the memory context is destroyed.
Anyway, perhaps the concerns of many partitions and the recursive
nature of these checks overrides that, because, as you say, possible
high memory usage.

Regards,
Greg Nancarrow
Fujitsu Australia

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2021-01-22 01:54:28 Re: OpenSSL connection setup debug callback issue
Previous Message Tomas Vondra 2021-01-22 01:26:29 Re: PoC/WIP: Extended statistics on expressions