Memory leak with CREATE TEMP TABLE ON COMMIT DROP?

From: Eric Ridge <e_ridge(at)tcdi(dot)com>
To: "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: Memory leak with CREATE TEMP TABLE ON COMMIT DROP?
Date: 2014-06-12 23:17:05
Message-ID: 165E6919-697C-4C50-9EEE-38728AC6D982@tcdi.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

# select version();
version
-----------------------------------------------------------------------------------------------------------------------------------
PostgreSQL 9.3.4 on x86_64-apple-darwin13.2.0, compiled by Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn), 64-bit
(1 row)

As best I can guess, Postgres has some kind of memory leak around (at least) temporary tables flagged to drop on commit. It's fairly easy to reproduce:

Terminal A
--------------
$ createdb leak
$ for i in $(seq 1 1000000) ; do echo "begin; create temp table foo() on commit drop; commit;"; done | psql leak > /dev/null

Terminal B
--------------
$ while(true); do ps auwx | grep $PID_OF_POSTGRES_PROCESS_FROM_TERMINAL_A; sleep 1 ; done

And watch the RSS size continue to climb, fairly quickly. This happens on both OS X and Linux (both x86_64).

We ran into this thanks to an update trigger that created a temporary table with on commit drop where we were literally updating millions of rows as atomic transactions, across about 100 concurrent connections, firing the trigger for each atomic update. The server quickly ran out of memory.

It took some time to find what appears to be the actual problem, but I think this is it. We've since rewritten the trigger to avoid using a temporary table (probably a good thing anyways) and all is well, but I was very shocked to see Postgres behaving badly here.

Any thoughts? And thanks for your time!

eric

PROPRIETARY AND COMPANY CONFIDENTIAL COMMUNICATIONS
The information contained in this communication is intended only for
the use of the addressee. Any other use is strictly prohibited.
Please notify the sender if you have received this message in error.
This communication is protected by applicable legal privileges and is
company confidential.

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Si Chen 2014-06-12 23:24:09 Re: what does pg_activity mean when the database is stuck?
Previous Message Merlin Moncure 2014-06-12 22:55:09 Re: max_connections reached in postgres 9.3.3