Re: psql blows up on BOM character sequence

On 03/25/2014 07:05 AM, Tom Lane wrote:
> Jim Nasby <jim(at)nasby(dot)net> writes:
>> Wait... I thought that was one of the objections... that we wanted to
>> leave a BOM in something like a COPY untouched?
>
> I think most of us are okay with stripping a BOM that appears at the
> *beginning* of a text file (assuming there's reason to believe the file
> is in UTF8 encoding). BOM sequences embedded later in the file are a lot
> more debatable, and I for one don't want to assume those can be dropped.
> I don't know of any legitimate usage of such cases, and think it's
> probably better to report an encoding error.

Yep, it's absolutely an encoding error IMO.

Something like:

ERROR: Invalid UTF-8 - probable UTF-8 byte-order mark detected
mid-data-stream.
HINT: Multiple files with byte-order marks were probably concatenated
with a tool that is not Unicode-aware.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services