Re: Backend crash (signal 11) in pg_trgm makesign() after ALTER TABLE ... SET STORAGE on a column with a gist_trgm_ops index

пт, 2 окт. 2026 г. в 11:07, Thiago Bonfante <thiago(at)subbase(dot)io>:

> Hello,
>
> We found that after *ALTER TABLE ... ALTER COLUMN ... SET STORAGE
> EXTENDED* (or MAIN) on a text column that has a
> gist_trgm_ops index, the next INSERT or non-HOT UPDATE into the table
> crashes the backend with signal 11.
> It reproduces every time on a stock installation.
>
> *== Version ==*
>
> PostgreSQL 18.6 (Debian 18.6-1.pgdg13+2) on aarch64-unknown-linux-gnu,
> compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit
>
> Also reproduced on 17.11 and 16.15 (same Debian PGDG packages), and first
> seen on Amazon Aurora PostgreSQL 16.11 and Amazon RDS PostgreSQL 16.13.
>
> *== Platform ==*
>
> Official postgres:18 Docker image, Debian GNU/Linux 13 (trixie), glibc
> 2.41 (Debian GLIBC 2.41-12+deb13u4)
> Kernel: Linux 6.12.76-linuxkit aarch64 (Docker Desktop VM on Apple
> Silicon), 11 CPUs, 8 GB RAM
> Configuration: the image's defaults; nothing changed in postgresql.conf,
> no extra start-up options.
>
> *== Steps to reproduce (psql, attached as repro.sql) ==*
>
> CREATE EXTENSION pg_trgm;
> CREATE TABLE t (id int, s text);
> INSERT INTO t SELECT g, 'item ' || g || ' ' || left(md5(g::text), 8) FROM
> generate_series(1, 10000) g;
> CREATE INDEX t_s_gist ON t USING gist (s gist_trgm_ops);
> SELECT attstorage FROM pg_attribute WHERE attrelid = 't_s_gist'::regclass;
> ALTER TABLE t ALTER COLUMN s SET STORAGE EXTENDED;
> SELECT attstorage FROM pg_attribute WHERE attrelid = 't_s_gist'::regclass;
> INSERT INTO t SELECT g, 'new item ' || g FROM generate_series(10001,
> 11000) g;
>
> *== Actual output ==*
>
> psql (with \set VERBOSITY verbose):
>
> CREATE EXTENSION
> CREATE TABLE
> INSERT 0 10000
> CREATE INDEX
> attstorage
> ------------
> p
> (1 row)
>
> ALTER TABLE
> attstorage
> ------------
> x
> (1 row)
>
> Server closed the connection unexpectedly
> This probably means the server terminated abnormally before or while
> processing the request.
> Connection to server was lost.
>
> *Server log:*
>
> LOG: client backend (PID 129) was terminated by signal 11: Segmentation
> fault
> DETAIL: Failed process was running: INSERT INTO t SELECT g, 'new item ' ||
> g FROM generate_series(10001, 11000) g;
> LOG: terminating any other active server processes
> LOG: all server processes terminated; reinitializing
>
> *== Expected output ==*
>
> The final INSERT should succeed (INSERT 0 1000). SET STORAGE EXTENDED does
> not change the storage of a text
> column (EXTENDED is already its default), and I would expect it to leave
> the index column alone: the
> index column's type is gtrgm, whose typstorage is 'p', and a newly created
> index on the same column gets 'p'.
>
> *== Observations ==*
>
> - SET STORAGE MAIN crashes the same way (index column p -> m). SET STORAGE
> PLAIN does not crash (p -> p).
> - REINDEX or DROP + CREATE INDEX sets the index column back to 'p' and the
> crash stops.
> - With core's tsvector GiST opclass (key type gtsvector, typstorage 'p'),
> SET STORAGE EXTENDED also changes
> the index column to 'x', but inserts do not crash.
>
> Backtrace on 16.13 with the PGDG debug symbols (postgresql-16-dbgsym):
>
> #0 makesign (sign=..., a=0xaaaaf41f9498, siglen=12) at
> contrib/pg_trgm/trgm_gist.c:109
> len = 44914044
> #1 gtrgm_penalty (fcinfo=...) at contrib/pg_trgm/trgm_gist.c:722
> #3 gistpenalty () at src/backend/access/gist/gistutil.c:733
> #4 gistchoose () at src/backend/access/gist/gistutil.c:458
> #5 gistdoinsert () at src/backend/access/gist/gist.c:749
> #6 gistinsert () at src/backend/access/gist/gist.c:184
> #7 ExecInsertIndexTuples () at src/backend/executor/execIndexing.c:432
>
> In another run, the crash was in unionkey() at
> contrib/pg_trgm/trgm_gist.c:555.
>
> Bytes of the key being inserted (gdb: x/12xb a, in makesign):
>
> b9 01 20 20 31 20 20 32 20 20 70 20
>
> The first byte (0xb9) is a 1-byte varlena header (length 92), followed by
> the ARRKEY flag (0x01) and the
> trigrams " 1", " 2", " p". Read with a 4-byte header, as the TRGM macros
> do, the length is
> (0x202001b9 >> 2) and the flag byte is 0x31, which matches len = 44914044
> above.
>
> *== Where this seems to come from ==*
>
> SetIndexStorageProperties() in src/backend/commands/tablecmds.c (current
> master) sets attstorage on every
> index column whose indkey references the altered column, without comparing
> the index column's type with
> the table column's type. With attstorage 'x' or 'm' on the gtrgm column,
> the new index tuple gets a short
> varlena header, and gtrgm_decompress() passes it on unchanged (it uses
> DatumGetTextPP).
>
> I hope this helps.
> Best,
>
>
>
> *Thiago Bonfante*
> Director of Engineering
> <eric(at)subbase(dot)io>
> <https://www.subbase.io/>
> <+19546847379>
> <https://www.instagram.com/subbase.io/>
>
> thiago(at)subbase(dot) <thiago(at)subbase(dot)io>io
> subbase.io <https://www.subbase.io/>
> +55 (45) 98811 5410 <+5545988115410>
> <https://www.linkedin.com/company/subbase/>
> <https://www.subbase.io/>
>

Hi, Thiago!

Thanks for the report!

ALTER TABLE ... SET STORAGE updates simple index columns through
SetIndexStorageProperties(). That copied the heap attstorage even when
ConstructTupleDescriptor() had replaced the index column type with the
opclass STORAGE type.

gist_trgm_ops stores gtrgm, whose typstorage is PLAIN. After SET STORAGE
EXTENDED the gist attstorage became EXTENDED. Later inserts packed short
varlena headers. TRGM macros use VARSIZE on a four-byte header and the
backend crashed.

Skip the catalog update when the index column type differs from the heap
column. A btree index on the same column still receives the new setting.
SET COMPRESSION uses the same helper.

--
Regards,
Rachitskiy Andrey