Skip site navigation (1) Skip section navigation (2)

Re: Troubles dumping a very large table.

From: "Merlin Moncure" <mmoncure(at)gmail(dot)com>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Ted Allen" <tallen(at)blackducksoftware(dot)com>, "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: Re: Troubles dumping a very large table.
Date: 2008-12-26 19:50:32
Message-ID: b42b73150812261150k6b17b819i704b6b7720243eb8@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-performance
On Fri, Dec 26, 2008 at 12:38 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Ted Allen <tallen(at)blackducksoftware(dot)com> writes:
>> 600mb measured by get_octet_length on data.  If there is a better way to measure the row/cell size, please let me know because we thought it was the >1Gb problem too.  We thought we were being conservative by getting rid of the larger rows but I guess we need to get rid of even more.
>
> Yeah, the average expansion of bytea data in COPY format is about 3X :-(
> So you need to get the max row length down to around 300mb.  I'm curious
> how you got the data in to start with --- were the values assembled on
> the server side?

Wouldn't binary style COPY be more forgiving in this regard?  (if so,
the OP might have better luck running COPY BINARY)...

This also goes for libpq traffic..large (>1mb) bytea definately want
to be passed using the binary switch in the protocol.

merlin

In response to

Responses

pgsql-performance by date

Next:From: Tom LaneDate: 2008-12-26 20:18:50
Subject: Re: Troubles dumping a very large table.
Previous:From: Tom LaneDate: 2008-12-26 17:38:38
Subject: Re: Troubles dumping a very large table.

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group