Re: BUG #5779: Large INSERT transaction with FOREIGN KEY constraints exhausts available RAM

On Wed, Dec 1, 2010 at 3:09 AM, Michel Alexandre Salim
<michel(dot)salim(at)cs(dot)fau(dot)de> wrote:
> I tried changing the fkey constraints to DEFERRABLE hoping that that would
> consume less RAM, but the same result occurs. Why should the memory usage
> pattern be different when integrity checks are done as part of the
> transaction (even when pushed back to the end), and in a separate
> transaction?

I'm just guessing here, but it may be that the trigger queue is what's
filling up all the memory. I'm guessing that a trigger event is
getting queued for each row you INSERT. But when you add the foreign
key later, it does a bulk validation of the entire table instead of
validating each individual row.

If that really is what's going on here, it's a known shortcoming of
the existing implementation which, unfortunately, no one has gotten
around to fixing (partly because it's not entirely obvious what the
design should be).

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company