Re: CopyReadLineText optimization

"Heikki Linnakangas" <heikki(at)enterprisedb(dot)com> writes:
> Andrew Dunstan wrote:
>> We currently make the following assumption in the code:
>>
>> * These four characters, and the CSV escape and quote characters, are
>> * assumed the same in frontend and backend encodings.
>>
>> The four characters are the carriage return, line feed, backslash and dot.
>>
>> I think the requirement might well actually be somewhat stronger than
>> that: i.e. that none of these will appear as a non-first byte in any
>> multi-byte client encoding character.

> No, we don't require that, and we do handle it correctly.

I believe we *have to* handle it correctly, because backslash actually
does appear as a non-first byte in SJIS or some other Far Eastern
encoding. In the CSV case such a restriction would be untenable anyway.

BTW, I notice that the code allows CSV escape and quote characters that
have the high bit set (in single-byte server encodings that is). Is
this a good idea? It seems like such are extremely unlikely to be the
same in two different encodings. Maybe we should restrict to the ASCII
range? Especially if the client encoding is multibyte ...

regards, tom lane