Fixing the libxml memory allocation situation

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-hackers(at)postgreSQL(dot)org
Subject: Fixing the libxml memory allocation situation
Date: 2009-05-12 21:55:13
Message-ID: 10685.1242165313@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I've been poking at the problems described here:
http://archives.postgresql.org/message-id/20090306191404.GK3901@alvh.no-ip.org
http://archives.postgresql.org/message-id/5265.1240417987@sss.pgh.pa.us

I've about come to the conclusion that the only real fix is to abandon
our attempt to manage libxml's memory usage. Although clearing out
whatever it's allocated at transaction end is safe from Postgres' own
point of view, we cannot really assume that third-party code that
also depends on libxml will think the same. What we have to do instead
is let it use malloc() directly and add PG_TRY() hacks as necessary to
ensure that transient data structures get cleared on error exits.

This has a number of disadvantages of course:

1. We are exposed to any internal memory leak or failure-to-check-for-null-
return-from-malloc bugs that may lurk inside libxml. However, it's hard
to argue that introducing known bugs of our own is a good tradeoff for
hiding someone else's possible bugs. We should take the attitude that
a libxml bug is a libxml bug, not our problem.

2. Any cleanup we miss making will result in a long-term memory leak
that accumulates across transactions. However, any such cleanup that
we are missing already represents an intratransaction leak, which will
build up with just as much vigor if the XML functions are called
repeatedly in one transaction. So it's not clear that the current
solution is buying much on this score either.

3. The code in xml.c will be a lot uglier :-(. We have a prototype,
which is the coding that was used in xml.c during most of 8.3 beta.
It's not real fun but at least the changes are localized.

This is a sufficiently large change that I'd not risk back-porting it
into 8.3, but I think it is the way to proceed for 8.4 and beyond.

Objections, better ideas?

regards, tom lane

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Олег Царев 2009-05-12 22:13:38 Re: Implementation of GROUPING SETS (T431: Extended grouping capabilities)
Previous Message Pavel Stehule 2009-05-12 21:20:14 Re: Implementation of GROUPING SETS (T431: Extended grouping capabilities)