BUG #19831: convert() between single-byte Cyrillic encodings returns wrong characters (KOI8R<->WIN866, WIN866->W

The following bug has been logged on the website:

Bug reference: 19831
Logged by: chunling qin
Email address: 303677365(at)qq(dot)com
PostgreSQL version: 18.6
Operating system: Linux x86_64 (official Docker image postgres:lates
Description:

**Description:**

`convert()` between two single-byte Cyrillic encodings takes a *direct*
conversion path (a hand-written per-byte table registered in
`pg_conversion`), while `convert_to()`/`convert_from()` always go through
UTF-8 using the Unicode-derived generated maps. For several encoding pairs
the two paths disagree: `convert()` silently returns a *different
character*, returns a byte for a character that PostgreSQL itself says has
no representation in the target encoding, or errors out for a character that
is representable.

This is not limited to the `convert()` function: the same direct conversions
are used for automatic client/server encoding conversion and for `COPY ...
ENCODING`. On a database created with `ENCODING 'KOI8R'`, I verified that
`SET client_encoding='WIN866'` returns corrupted results and that `COPY FROM
... ENCODING 'WIN866'` writes corrupted bytes to disk (details available on
request).

**Reproduction (self-contained, works on the official image):**

```sh
docker exec -i <postgres:latest container> psql -U postgres -X -P pager=off
<<'SQL'
-- KOI8-R byte 0xa4 is U+2553 (BOX DRAWINGS DOUBLE VERTICAL AND HORIZONTAL)
SELECT convert_from(decode('a4','hex'), 'KOI8R') AS
src_char;
SELECT encode(convert(decode('a4','hex'),'KOI8R','WIN866'), 'hex') AS
direct;
SELECT encode(convert_to(convert_from(decode('a4','hex'),'KOI8R'),
'WIN866'), 'hex') AS
two_step_via_utf8;

-- a longer string, showing silent character substitution:
SELECT convert_from(decode('a441a6','hex'), 'KOI8R')
AS original,
convert_from(convert(decode('a441a6','hex'),'KOI8R','WIN866'),
'WIN866') AS converted;
SQL
```

**Actual output:**

```
src_char | direct | two_step_via_utf8
----------+--------+--------------------
╓ | f3 | d6

original | converted
----------+-----------
╓A╕ | єA∙
```

`f3` in WIN866 is U+0454 (CYRILLIC SMALL LETTER UKRAINIAN IE) — a different
character. The two-step path through PostgreSQL's own UTF-8 maps yields
`d6`, which is the correct byte for U+2553 in WIN866.

**Full matrix (all verified on the official 18.6 image):**

| Conversion | Input | convert() returns | Correct (per PG's own UTF-8 maps)
|
|---|---|---|---|
| KOI8R -> WIN866 | a4 (U+2553) | f3 = є U+0454 | d6 |
| KOI8R -> WIN866 | a6 (U+2555) | f9 = ∙ U+2219 | b8 |
| KOI8R -> WIN866 | a7 (U+2556) | f5 = ї U+0457 | b7 |
| KOI8R -> WIN866 | ad (U+255C) | ad = н U+043D | bd |
| KOI8R -> WIN866 | b4/b6/b7/bd | f2/f8/f4/bd (wrong chars) | b6/d1/d2/d7 |
| WIN866 -> KOI8R | bd/f8/f9 | bd/b6/a6 (wrong chars) | ad/9c/95 |
| WIN866 -> WIN1251 | f8 (° U+00B0) | b2 = І U+0406 | b0 |
| KOI8R -> WIN1251 | a4 (U+2553) | **ba (no error!)** | should raise 22P05
(U+2553 has no WIN1251 representation; PG's own convert_to() does error) |
| ISO_8859_5 -> WIN1251 | a0 (U+00A0 NBSP) | **ERROR 22P05** | a0 (WIN1251
does have NBSP at 0xa0; PG's own convert_to() succeeds) |
| LATIN2 -> WIN1250 | 80 (U+0080) | **80 passed through = € U+20AC** |
should raise 22P05 (U+0080 has no WIN1250 representation; PG's own
convert_to() does error) |

So the same bug family produces three distinct symptoms:
1. **Silent character substitution** (wrong result / data corruption),
2. **Missing error** where PostgreSQL's own conversion errors,
3. **False error** where PostgreSQL's own conversion succeeds.

**Root cause:**

`src/backend/utils/mb/conversion_procs/cyrillic/cyrillic.c` contains
hand-written 128-entry tables (`koi2win866`, `win8662koi`, `koi2win1251`,
`win12512koi`, `iso2win1251`, ...) consumed by `local2local()`
(`src/backend/utils/mb/conv.c:33`), which indexes them with the low 7 bits
of each byte. At least some entries disagree with the Unicode-derived
generated maps (`src/backend/utils/mb/Unicode/*`). For example, in
`koi2win866[]` the entry for index 0x24 (i.e. input byte 0xa4) is `0xF3`,
while composing PG's own `koi8r_to_utf8` + `utf8_to_win866` maps requires
`0xD6`.

The tables appear to be historical hand-written data that was never audited
against the Unicode-derived maps. The regression test
`src/test/regress/sql/conversion.sql` covers only `iso8859-5 -> koi8r` with
a few inputs (ASCII, Cyrillic letters, embedded NUL) and does not exercise
the pseudographics byte range 0x80-0xBF where the disagreements live, which
is presumably why this has gone unnoticed.

**Expected behavior:** a direct conversion between two encodings should
produce the same result as going through UTF-8 with PostgreSQL's own
generated maps (or, where the direct table is believed to be authoritative,
the generated maps and the direct table should at least agree with each
other and with the documented encoding semantics). If maintaining per-pair
tables is not considered worthwhile, routing single-byte <-> single-byte
conversions through UTF-8 would remove the inconsistency.

**Impact:** silent data corruption for `convert()`; on non-UTF8 databases
the same code path serves automatic client_encoding conversion and `COPY ...
ENCODING`, so imported/exported data can be silently corrupted (verified on
a KOI8R database for `COPY FROM ... ENCODING 'WIN866'`: input byte 0xBD =
U+255C was stored as 0xBD = U+256B).

**Environment / determinism:** no special settings or extensions; results
are deterministic across repeated runs; reproduced identically when the
input comes from a table column instead of a literal, and under
`PREPARE`/`EXECUTE` with `plan_cache_mode=force_generic_plan`, so this is
not constant-folding related.