Re: [HACKERS] memory problems in copying large table to STDOUT

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Martin Weinberg <weinberg(at)osprey(dot)astro(dot)umass(dot)edu>
Cc: pgsql-hackers(at)postgreSQL(dot)org
Subject: Re: [HACKERS] memory problems in copying large table to STDOUT
Date: 1999-10-09 15:54:34
Message-ID: 11327.939484474@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Martin Weinberg <weinberg(at)osprey(dot)astro(dot)umass(dot)edu> writes:
> I've been struggling to copy a large table (200 million
> records, 60GB) to tape using:
> psql -qc "copy psc to STDOUT;" Winter99 | dd of=/dev/st0 bs=32k
> After processing about 10 million records (this varies), I
> get:
> FATAL 1: Memory exhausted in AllocSetAlloc()

Hmm. What is the exact declaration of the table?

The only explanation I can think of offhand is that the output
conversion function for one of the column types is leaking memory...
copy.c itself looks to be pretty careful not to.

regards, tom lane

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 1999-10-09 18:42:56 Re: [HACKERS] memory problems in copying large table to STDOUT
Previous Message Tom Lane 1999-10-09 15:49:33 Re: [HACKERS] Re: [GENERAL] Update of bitmask type