Re: pg_read_file() and non-ascii input file

From: Takahiro Itagaki <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_read_file() and non-ascii input file
Date: 2010-01-04 02:10:07
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> > If we want to keep backward compatibility, the issue can be fixed
> > by adding pg_verifymbstr() to the function.
> I don't feel good about changing the return type of an existing
> function, so I guess +1 from me on the approach quoted above.

Ok, I just added pg_verifymbstr() instead of changing the result type.

I didn't add any additinal file reading functions in the patch, but
I'm willing to add them if someone want them:
- pg_read_file_with_encoding()
- pg_read_binary_file() RETURNS bytea
- pg_read_text_file() RETURNS SETOF text -- returns set of lines

One thing bothering me is the HINT message on error is just pointless;
The encoding is controlled by "server_encoding" here. We will have the
same error message in server-side COPY commands. We'd better improving
the message, though it should be done by another patch.

=# SELECT pg_read_file('invalid.txt', 0, (pg_stat_file('invalid.txt')).size);
ERROR: invalid byte sequence for encoding "UTF8": 0x93
HINT: This error can also happen if the byte sequence does not match the
encoding expected by the server, which is controlled by "client_encoding".

=# COPY tbl FROM 'invalid.txt'; -- server-side copy from a local file.
(the same message -- but the encoding should match with "server_encoding")

Takahiro Itagaki
NTT Open Source Software Center

Attachment Content-Type Size
pg_read_file_20100104.patch application/octet-stream 785 bytes

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2010-01-04 02:44:40 pgsql: When estimating the selectivity of an inequality "column >
Previous Message Takahiro Itagaki 2010-01-04 01:34:42 Re: Verifying variable names in pgbench