Re: Memory use in 8.3 plpgsql with heavy use of xpath()

"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

> Well, you tell me --- *you* reported a behavior that isn't obviously
> explained by the bug we found.

In case it wasn't clear, the bug found was a intra-transaction memory leak.
When the transaction ended the memory would be reclaimed. That doesn't seem to
match the description of long term memory growth.

> It's possible that what you were seeing was an indirect effect of the
> now-known bug: if the xpath leak were to occur repeatedly on a large
> scale in a long-lived session, I think it's possible that memory
> allocation behavior might suffer due to fragmentation effects.
> I feel that that's a pretty hand-wavy explanation though.

Another explanation is that there wasn't a inter-transaction memory leak, it
was just that the high water mark would grow whenever a transaction processed
more data than previous transactions. If the data set size varies a lot most
common distributions would have the majority of data sets be about the same
size with a long tail of larger sets. That might like a slow creep as the
increases get rarer and rarer but do continue to happen.

That's just a special case of what would be expected to happen with memory
allocation anyways though. Few allocators return memory to the OS anyways. It
might just be exaggerated in this case since probably a significant part of
Postgres's footprint here was the per-transaction memory being used by this
leak.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training!