Problem with large bytea fields in windows

From: "Al Rosenthal" <arosenthal(at)AtlantaHand(dot)com>
To: <pgsql-general(at)postgresql(dot)org>
Subject: Problem with large bytea fields in windows
Date: 2007-07-18 01:51:15
Message-ID: 000001c7c935$7e276870$b900a8c0@Tablet
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I have been having a lot of trouble with postgres and large bytea fields
under windows and would appreciate any suggestions.

I have a large database (22 GB) that I have slowly grown on one machine. It
is currently running postgres 8.2.3 under windows xp. It is a simple db
with one table that has a serial column and a large bytea column. It is a
repository for scanned documents. A few of these documents are over 250
pages and take up at least 100 Mb per row. A backup takes almost a whole
day and almost as long to copy onto usb drive. I have not had any trouble
with these large inserts (using libpq calls) or in making a backup.

Recently I started using slony, first for a 400 MB database, and then I
attempted to use it on this large database. The slave machine is running
windows vista and the same version of postgres. The copy failed at about
12GB with a windows error 10055, ran out of buffer space. I have now tried
over 30 times, all without success. I tried using just two machines sitting
next to each other and an ethernet switch. I tried adjusting the tcp
parameters in the registry following microsoft's advice in a few of the
knowledgebase articles. I tried running the slony process on a third linux
machine. Always the same problem. I ran the process with no one logged
onto these machines so no other processess were running. I shut off all
unecessary services. Stil the copies failed after 12GB to 15GB. Same
error. I turned off autovacuum, I increase all of the memory configs in
postgres.conf. I added memory to both machines so they both have 2GB.

I posted this problem to the slony.info list. The main suggestion seems to
be that it is really a libpq error. Darcy Buskermolen suggested that the
problem was with libpq since the initial subscribe process mimics a pg_dump
| psql. He suggested a postgres mailing list post:
http://archives.postgresql.org/pgsql-ports/2005-11/msg00000.php, which
describes a winsock problem in libpq. No resolution is given howerver.

Does anyone have any suggestions? At the moment I am all out of ideas.

Thank you for any help

Al Rosenthal
arosnethal at AtlantaHand dot com

Browse pgsql-general by date

  From Date Subject
Next Message Zlatko Matić 2007-07-18 06:24:38 protect a database
Previous Message Alvaro Herrera 2007-07-18 01:26:32 Re: AutoVacuum Behaviour Question