Re: [HACKERS] [POC] Faster processing at Gather node

From: Rafia Sabih <rafia(dot)sabih(at)enterprisedb(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, PostgreSQL Developers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [HACKERS] [POC] Faster processing at Gather node
Date: 2017-12-05 13:02:52
Message-ID: CAOGQiiN6p80jYCZC_N49LwWKCxvs8-=K1LD1nv3c7aQr-ULYVA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Dec 4, 2017 at 9:20 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On Sun, Dec 3, 2017 at 10:30 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
> > I thought there are some cases (though less) where we want to Shutdown
> > the nodes (ExecShutdownNode) earlier and release the resources sooner.
> > However, if you are not completely sure about this change, then we can
> > leave it as it. Thanks for sharing your thoughts.
>
> OK, thanks. I committed that patch, after first running 100 million
> tuples through a Gather over and over again to test for leaks.
> Hopefully I haven't missed anything here, but it looks like it's
> solid. Here once again are the remaining patches. While the
> already-committed patches are nice, these two are the ones that
>

Hi,
I was spending sometime in verifying this memory-leak patch for
gather-merge case and I too found it good. In the query I tried, around 10
million tuples were passed through gather-merge. On analysing the output of
top it looks acceptable memory usage and it gets freed once the query is
completed. Since, I was trying on my local system only, I tested for upto 8
workers and didn't find any memory leaks for the queries I tried.
One may find the attached file for the test-case.

--
Regards,
Rafia Sabih
EnterpriseDB: http://www.enterprisedb.com/

Attachment Content-Type Size
ml_test_query.sql application/octet-stream 1021 bytes

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2017-12-05 14:01:35 Speeding up pg_upgrade
Previous Message Fabien COELHO 2017-12-05 13:00:09 Re: [HACKERS] pow support for pgbench