[PG19] psql backslash commands fail with translated messages and a non UTF-8 client encoding

From: shihao zhong <zhong950419(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Greg Sabino Mullane <htamfids(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
Subject: [PG19] psql backslash commands fail with translated messages and a non UTF-8 client encoding
Date: 2026-10-13 04:26:07
Message-ID: CAGRkXqRv7yhBOhF0YvSr6ZB7rvsPCjnXSPj8=3bKcSJuYvNrBA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

This is Medium 6 from my list in [1].

Since 41d69e6dcca, psql puts a label in a comment in front of its
hidden queries. The label is translated, so it is in the encoding of
the message locale. The query is sent in client_encoding. When the two
differ the server rejects the query.

createdb -E EUC_JP -l C -T template0 eucjp
LANG=ja_JP.UTF-8 psql -c '\dn' eucjp | cat
ERROR: invalid byte sequence for encoding "EUC_JP": 0xe3 0x81

This needs a psql built with NLS. When the output is not a terminal,
psql does not take client_encoding from the locale, so it is the
database encoding. That makes scripts and cron jobs the ones that
break. \d, \dt, \l, \du, \dx, \sf and \sv fail the same way. psql 18
works against the same server.

I think this is a very disappointing behavior for non UTF-8 users such
as Japan and China.

0001 sends the query without the label when the label is not plain
ASCII and the locale encoding differs from client_encoding. In every
other case the label is sent as before, and -E always shows it.

I cant write a test for it as I do not have NLS and a Japanese locale.

A complete fix would convert the label to client_encoding, so that
the server log gets it in every case. For me, that looks like a project
for 20.

I can try that one if people want a complex fix

The commit came from [2].

[1]
https://postgr.es/m/CAGRkXqSFdjqZP_kwRz7dv6LHX-CSrkDnynNXgntRsCEnQTKxmg@mail.gmail.com
[2]
https://postgr.es/m/CAKAnmmJz8Hh=8Ru8jgzySPWmLBhnv4=oc_0KRiz-UORJ0Dex+w@mail.gmail.com

Thanks,
Shihao

Attachment Content-Type Size
v1-0001-psql-Don-t-send-hidden-query-labels-the-server-co.patch application/octet-stream 3.1 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2026-10-13 04:31:39 Re: [PG19] pg_dump --statistics restores extended statistics by attribute number
Previous Message Fujii Masao 2026-10-13 04:21:30 Re: [PATCH] Fix pg_dump emitting OVERRIDING SYSTEM VALUE for tables with dropped identity columns