Re: BUG #16148: Query on Large table hangs in ETL flows and gives out of memory when run in pgAdmin4

From: Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
To: Scott Volkers <scottvolkers(at)gmail(dot)com>
Cc: pgsql-bugs <pgsql-bugs(at)lists(dot)postgresql(dot)org>
Subject: Re: BUG #16148: Query on Large table hangs in ETL flows and gives out of memory when run in pgAdmin4
Date: 2019-12-05 16:52:06
Message-ID: CAMkU=1yQnwmdtUGYkmC7rm2H7e0ChTcZWr_LR7NzurH6dxhTUw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

On Thu, Dec 5, 2019 at 9:54 AM Scott Volkers <scottvolkers(at)gmail(dot)com> wrote:

> Hi Jeff,
>
> I may not have explained this well.
>
> The long and the short of it is this where clause
> FROM "elliedb"."documentlog" WHERE dcmodifiedutc>(extract(epoch from
> TIMESTAMP '2019-11-15 11:30:51')*1000)
>
> causes and out of memory error in PGAdmin. The query will not run. I am
> testing it there because the same query will not run in Informatica ETL
> task flow. It hangs our processes.
>

WHERE clauses do not get executed. Whole statements do. We can't verify
or investigate a bug report based on non-executable fragments. I can just
make up a "select count(*)" to go in front, but when I do I observe no bug.

Based on the information provided, the most likely explanation is that you
are trying to read a huge result set into client memory, and you don't have
enough memory to do so. If you want to manipulate a huge result in
pgAdmin4 or Informatica without reading them into client memory, then that
is not a bug in PostgreSQL, it is question about pgAdmin4 or Informatica.

Cheers,

Jeff

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Mark Phillips 2019-12-05 19:52:10 Re: Create database from template very slow
Previous Message PG Bug reporting form 2019-12-05 16:02:33 BUG #16153: foreign key update should probably move dependent rows in the case of tuple rerouting