COPY problems with psql / libpq

From: "Oliver Elphick" <olly(at)lfix(dot)co(dot)uk>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: COPY problems with psql / libpq
Date: 2000-01-20 15:21:24
Message-ID: 200001201521.PAA04675@linda.lfix.co.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

[Using current cvs] I have a problem with COPY when called like this:

psql -e bray </tmp/ol

with one particular file, containing commands and data like this:

========================================
copy address from stdin;
1 Some place, Regal Way Somewhere Oxon AB1 3CF [ Tel: 01367 888888 ] No
Deliveries Pm Fridays GB \N \N \N \N \N \N \N \N \N \N \N \N
...
1000 73 Some Road London SW1 1ZZ GB 44 81 999 9999 \N
\N \N \N \N \N \N \N
\.
-- 1000 records written

select count(*) from address;

copy address from stdin;
1001...
... and so on up to 3916 records in total, divided into 1000 record chunks
========================================

psql or libpq seems to choke on the data, so that some spurious error
arises, such as null input into a non-null field. Thereafter, libpq
seems to get stuck in a COPY state:

copy address from stdin;
-- 1000 records written
select count(*) from address;
PQexec: you gotta get out of a COPY state yourself.

(The comment in PQexec says it does this to preserve backwards
compatibility, but getting stuck in COPY state is not backwards
compatible!)

If I remove the SQL commands from the input file, go into psql and do:

copy address from '/tmp/ol';

all 3916 records are added correctly. This seems to indicate that
the problem is not in the backend.

I found that if I broke the first 1000 records into 2 equal parts, all
of them were added correctly without error; so I conclude that data
is being buffered and lost somewhere in psql or libpq, and the problem is
dependent on the amount of data being copied.

This began to happen within the last week, but I don't know which
recent change is responsible.

--
Oliver Elphick Oliver(dot)Elphick(at)lfix(dot)co(dot)uk
Isle of Wight http://www.lfix.co.uk/oliver
PGP key from public servers; key ID 32B8FAA1
========================================
"Neither is there salvation in any other; for there is
none other name under heaven given among men, whereby
we must be saved." Acts 4:12

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Lockhart 2000-01-20 15:28:57 Re: [HACKERS] A notice for too long names
Previous Message Bruce Momjian 2000-01-20 15:17:04 Re: Postgres improvement