Skip site navigation (1) Skip section navigation (2)

Re: BUG #4996: postgres.exe memory consumption keeps going up

From: Craig Ringer <craig(at)postnewspapers(dot)com(dot)au>
To: "WANGRUNGVICHAISRI, SHIVESH" <sbw(at)appsig(dot)com>, PostgreSQL bugs <pgsql-bugs(at)postgresql(dot)org>
Subject: Re: BUG #4996: postgres.exe memory consumption keeps going up
Date: 2009-08-26 05:20:51
Message-ID: 4A94C633.2070802@postnewspapers.com.au (view raw or flat)
Thread:
Lists: pgsql-bugspgsql-hackers
On 26/08/2009 9:25 AM, WANGRUNGVICHAISRI, SHIVESH wrote:
> Hi,
>
> I have uploaded everything that you should need here:
>
> http://www.filefactory.com/file/ah334c6/n/Pgsql_Leak_zip

I notice that you used the Windows Task Manager to monitor memory usage, 
rather than Process Explorer. There's a reason I suggested that you 
monitor with Process Explorer: it provides more detail about the 
process's memory usage. In particular, it lets you tell how much of the 
memory usage is shared memory and how much is private.

The Task Manager does not. Because PostgreSQL makes heavy use of shared 
memory this information is important.

That said, the "VM size" looks like the private size, making the rest 
shared. The growth you're observing is in shared memory, suggesting it's 
not in fact a leak at all, but the backend touching more and more of 
shared_buffers .

I don't see any sign of a leak in postgres.exe . Given your 
configuration, I'd expect to see postgres.exe's memory use grow up to 
between 256 and say 290-ish MB (given that you have shared_buffers = 
256MB) and none of your screenshots show it exceeding that. It all looks 
quite normal to me from what you've posted.

You need to understand that PostgreSQL uses shared memory. The 
postmaster allocates this memory *once*. All postgres.exe backends 
attach to this shared memory block, which is already allocated. The way 
Windows accounts for memory use means that they don't immediately show 
up as using that memory, but it does get added to their reported memory 
usage as they read from and write to the shared memory block. Despite it 
being shown as part of each backend's memory use, the memory is actually 
ONLY USED ONCE - by the postmaster.

You haven't let the test run long enough for the crash you've reported 
to take place or even for postgres.exe to grow beyond normal size (if it 
does). You also haven't provided the contents of postgresql's log file 
after the crash you described. As a result so I can't tell what happens 
when growth hits the ~2GB limit you mentioned in your initial post.

I can't reproduce the crash myself since shared usage on my postgres.exe 
process(es) stabilizes and ceases growing after a while, as it should.

I'm running your test program on my sandbox machine now to see if I can 
reproduce the issue, but so far everything looks quite normal. Both 
active backends have stabilized at 37,965K of shared memory - one has a 
total working set of 41,697K and the other 41,620K (so each is using 
about 3.6MB of real system RAM). I'll come back tomorrow and see if 
there are any signs of anything odd going on, but right now it looks 
sensible.

SandBox.exe is growing in the same way my locally built test program is. 
Weird. I think I'm going to have to recompile PostgreSQL's libpq using 
VC++8 in memory debugging mode. Sigh.


--
Craig Ringer

In response to

Responses

pgsql-hackers by date

Next:From: Markus WannerDate: 2009-08-26 06:53:00
Subject: Re: setting up scan keys
Previous:From: KaiGai KoheiDate: 2009-08-26 02:39:03
Subject: Difficulty in partial writable TOAST value

pgsql-bugs by date

Next:From: Gergely CzuczyDate: 2009-08-26 06:29:40
Subject: BUG #5012: mailing list VERP addresses not sender-verifiable
Previous:From: Alvaro HerreraDate: 2009-08-26 04:34:54
Subject: Re: BUG #4999: select 'a' < 'A' is true, but should be false . . .

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group