Re: BUG #19525: In `contrib/dict_int`, handling a token whose first byte is a null byte causes `pnstrdup()` .

Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com> writes:
> On Thu, 18 Jun 2026 at 18:54, PG Bug reporting form <noreply(at)postgresql(dot)org>
> wrote:
>> -- 3. Create a C helper (raw_lexize.so) that invokes the lexize callback
>> with
>> -- a raw bytea token, bypassing the text encoding layer.
>> CREATE FUNCTION raw_lexize(dict regdictionary, token bytea)
>> RETURNS text[] AS 'raw_lexize', 'raw_lexize' LANGUAGE C STRICT;

> Your analysis seems right to me.

It is not. I do not think this is a bug; if it is a bug we have got
probably hundreds of places that'd have to be fixed. The general rule
in Postgres is that we do not permit embedded nulls in text values.
The fact that a misguided superuser can break that rule doesn't make
it a bug for code to depend on it. (There are approximately an
infinite number of ways for a misguided superuser to break Postgres,
just as the root user can destroy a Unix system many times over.)

So I see no value in this proposed change in dintdict_lexize.
But the one in dxsyn_lexize seems worthwhile, just because
it makes that code shorter, simpler, and faster. Since that's
mostly an example module, I don't feel a need to back-patch,
but I'll go commit that to HEAD.

regards, tom lane