Skip site navigation (1) Skip section navigation (2)

BUG #4204: COPY to table with FK has memory leak

From: "Tomasz Rybak" <bogomips(at)post(dot)pl>
To: pgsql-bugs(at)postgresql(dot)org
Subject: BUG #4204: COPY to table with FK has memory leak
Date: 2008-05-27 19:14:23
Message-ID: 200805271914.m4RJEN0r012867@wwwmaster.postgresql.org (view raw or flat)
Thread:
Lists: pgsql-bugspgsql-hackers
The following bug has been logged online:

Bug reference:      4204
Logged by:          Tomasz Rybak
Email address:      bogomips(at)post(dot)pl
PostgreSQL version: 8.3.1
Operating system:   Debian Linux
Description:        COPY to table with FK has memory leak
Details: 

I tried to use COPY to import 27M rows to table:
CREATE TABLE sputnik.sputnik (
        id INTEGER,
        sequence BIGINT,
        strength INTEGER,
        time TIMESTAMP WITH TIME ZONE,
        tags TEXT ARRAY[1]
);
CREATE TABLE sputnik.ccc24 (
        station CHARACTER(4) NOT NULL REFERENCES sputnik.station24 (id),
        moment INTEGER NOT NULL,
        flags INTEGER NOT NULL
) INHERITS (sputnik.sputnik);
COPY sputnik.ccc24(id, moment, station, strength, sequence, flags)
FROM '/tmp/24c3' WITH DELIMITER AS ' ';

When importing on amd64 with !G RAM and 1G swap it takes entire RAM+swap and
gets killed by OOM-killer.
After dividing data into three parts I was able to
import it.
Kernel kills server process, not psql.

On i386 with 512M RAM and !G swap it works, but
takes entire RAM and about 700MB of swap.
Amount of used swap changes, and oscilates between
600M and 900M for almost entire time of import.

So on 32-bit system, where pointers are smaller,
I was able to import, on 64-bit PostreSQL gets killed.

When I dropped FK from sputnik.ccc24, used COPY, and then created FK
everything was correct, and PostgreSQL didn't even touched swap.

In January, on 8.3rc2 I had also problem with the same data set; when I
tried to update:
UPDATE sputnik.ccc24 SET time = to_timestamp(moment);
my PostgreSQL server also was killed by OOM-killer.
Then patch from Tom Lane for ri_FetchConstraintInfo in
src/backend/utils/adt/ri_triggers.c (1.102->1.102) helped.

I think that here situation may be similar.
I even tried to write some patch:
--- src/backend/executor/execMain.c    2008-02-03 12:12:01.000000000 +0100
+++ src/backend/executor/execMain.c    2008-02-03 12:13:09.000000000 +0100
@@ -1905,6 +1905,7 @@
                        return check[i].ccname;
        }
 
+       FreeExprContext(econtext);
        /* NULL result means no error */
        return NULL;
 }

but after this patch 59 regression tests failed.

But I still think that some code responsible for checking constraints do not
frees usused memory.

Responses

pgsql-hackers by date

Next:From: Dave PageDate: 2008-05-27 19:18:29
Subject: Re: ERRORDATA_STACK_SIZE panic crashes on Windows
Previous:From: Magnus HaganderDate: 2008-05-27 19:09:16
Subject: Re: ERRORDATA_STACK_SIZE panic crashes on Windows

pgsql-bugs by date

Next:From: Joe ConwayDate: 2008-05-27 19:39:37
Subject: Re: BUG #4203: perform dblink() in begin/exception returns wrong SQLSTATE code
Previous:From: Tom LaneDate: 2008-05-27 15:56:43
Subject: Re: BUG #4203: perform dblink() in begin/exception returns wrong SQLSTATE code

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group