Re: index prefetching

Hi Peter,

I found a crash in v36-0005 with an INCLUDE-only index-only scan when
the SP-GiST opclass cannot return its key:

CREATE TEMP TABLE poly_include (id int, p polygon);
INSERT INTO poly_include VALUES
(7, polygon(box(point(0,0), point(1.0000000009313226,1))));
CREATE INDEX poly_include_idx ON poly_include
USING spgist (p) INCLUDE (id);
VACUUM (FREEZE, ANALYZE) poly_include;
SET enable_seqscan = off;
SET enable_bitmapscan = off;
SELECT id FROM poly_include;
-- server closed the connection unexpectedly

The server log has:

LOG: client backend was terminated by signal 11: Segmentation fault
DETAIL: Failed process was running: SELECT id FROM poly_include

SELECT id only needs the included id, so the planner uses an Index
Only Scan even though the polygon key itself cannot be returned. The
query returns 7 before 0005; with 0005, the backend crashes.

Here is the complete backtrace from the v36 core:

(gdb) bt
#0 0x00007f40c359a746 in __memmove_evex_unaligned_erms () from /lib64/libc.so.6
#1 0x0000000000540fde in fill_val (att=0x16b73d0, bit=<optimized
out>, bitmask=<optimized out>, dataP=0x7ffd3eaa8a68,
infomask=<optimized out>, datum=<optimized out>, isnull=false) at
heaptuple.c:367
#2 0x0000000000541efa in heap_fill_tuple
(tupleDesc=tupleDesc(at)entry=0x16b73b0,
values=values(at)entry=0x7ffd3eaa8b60,
isnull=isnull(at)entry=0x7ffd3eaa8b40, data=<optimized out>,
data(at)entry=0x7f40abc99078 "", data_size=data_size(at)entry=1048580,
infomask=infomask(at)entry=0x7f40abc99074, bit=0x0) at heaptuple.c:433
#3 0x00000000005429c1 in heap_form_tuple (tupleDescriptor=0x16b73b0,
values=0x7ffd3eaa8b60, isnull=0x7ffd3eaa8b40) at heaptuple.c:1095
#4 0x00000000005d2bb0 in spggettransform (scan=0x16b6d70,
batch=0x16d89e8, item=<optimized out>) at spgscan.c:1446
#5 0x0000000000591773 in heapam_index_set_scanpos_tid
(scan=0x16b6d70, hscan=0x16b74c0, direction=ForwardScanDirection,
scanBatch=0x16d89e8, scanPos=0x16b6da8, all_visible=0x7ffd3eaa8d0f) at
heapam_indexscan.c:1037
#6 0x00000000005932ec in heapam_index_getnext_scanbatch
(all_visible=0x7ffd3eaa8d0f, direction=ForwardScanDirection,
scan=0x16b6d70) at heapam_indexscan.c:1003
#7 heapam_index_getnext_slot (amgetbatch=true, index_only=true,
slot=0x16b6178, direction=ForwardScanDirection, scan=0x16b6d70) at
heapam_indexscan.c:574
#8 heapam_index_only_batch_getnext_slot (scan=0x16b6d70,
direction=ForwardScanDirection, slot=0x16b6178) at
heapam_indexscan.c:514
#9 0x000000000074c869 in table_index_getnext_slot (slot=0x16b6178,
direction=ForwardScanDirection, scan=0x16b6d70) at
../../../src/include/access/tableam.h:1383
#10 IndexOnlyNext (node=node(at)entry=0x16b5cf8) at nodeIndexonlyscan.c:112
#11 0x000000000072ed3c in ExecScanFetch (recheckMtd=0x4edb1e
<IndexOnlyRecheck>, accessMtd=0x74c7f0 <IndexOnlyNext>, epqstate=0x0,
node=0x16b5cf8) at ../../../src/include/executor/execScan.h:135
#12 ExecScanExtended (projInfo=0x16b63f8, qual=0x0, epqstate=0x0,
recheckMtd=0x4edb1e <IndexOnlyRecheck>, accessMtd=0x74c7f0
<IndexOnlyNext>, node=<optimized out>) at
../../../src/include/executor/execScan.h:196
#13 ExecScan (node=0x16b5cf8, accessMtd=0x74c7f0 <IndexOnlyNext>,
recheckMtd=0x4edb1e <IndexOnlyRecheck>) at execScan.c:59
#14 0x00000000007238e3 in ExecProcNode (node=0x16b5cf8) at
../../../src/include/executor/executor.h:327
#15 ExecutePlan (dest=0x16bd640, direction=<optimized out>,
numberTuples=0, sendTuples=<optimized out>, operation=CMD_SELECT,
queryDesc=0x1605920) at execMain.c:1766
#16 standard_ExecutorRun (queryDesc=0x1605920, direction=<optimized
out>, count=0) at execMain.c:377
#17 0x0000000000920fb8 in PortalRunSelect (portal=0x1660a50,
forward=<optimized out>, count=0, dest=<optimized out>) at
pquery.c:917
#18 0x00000000009224cb in PortalRun (portal=0x1660a50,
count=9223372036854775807, isTopLevel=<optimized out>, dest=0x16bd640,
altdest=0x16bd640, qc=0x7ffd3eaa9060) at pquery.c:761
#19 0x000000000091e1d2 in exec_simple_query (query_string=0x15dc790
"SELECT id FROM poly_include") at postgres.c:1297
#20 0x000000000091ff13 in PostgresMain (dbname=<optimized out>,
username=<optimized out>) at postgres.c:4946
#21 0x000000000091a27d in BackendMain (startup_data=<optimized out>,
startup_data_len=<optimized out>) at backend_startup.c:124
#22 0x0000000000864a7d in postmaster_child_launch
(child_type=<optimized out>, child_slot=1,
startup_data=0x7ffd3eaa94d0, startup_data_len=24,
client_sock=0x7ffd3eaa94f0) at launch_backend.c:268
#23 0x00000000008681f6 in BackendStartup (client_sock=0x7ffd3eaa94f0)
at postmaster.c:3640
#24 ServerLoop () at postmaster.c:1727
#25 0x0000000000869c6d in PostmasterMain (argc=<optimized out>,
argv=0x15d5d10) at postmaster.c:1414
#26 0x00000000005322c5 in main (argc=15, argv=0x15d5d10) at main.c:227

spggettransform() calls leaf_consistent with in.returnData=true, then
passes out.leafValue to heap_form_tuple() as the value of a POLYGON
column. The polygon opclass actually stores a BOX and has
canReturnData=false, so heap_form_tuple() treats that BOX as a POLYGON.
In frame #2, heap_fill_tuple() has data_size=1048580: it read the first
bytes of the 32-byte BOX as a POLYGON varlena length, then crashed in
memmove() while trying to copy that much data.

The old amgettuple path already has the same type mismatch. It also
reads past the 32-byte BOX, but in my run the BOX still pointed into a
shared buffer mapping and the read did not fault. Since the query only
projects the included id, the executor ignored the malformed polygon
value and returned 7. After 0005, spgLeafTest() copies the leaf tuple,
including the BOX, into batch->tuples at spgscan.c:663.
spggettransform() reads it back at spgscan.c:1397; the same over-read
runs past that allocation and crashes before the tuple can be returned.

The attached 0001 checks canReturnData before reconstructing the key.
When canReturnData is false, it sets the key column to NULL and still
returns the INCLUDE column. It also adds this case to
create_index_spgist.

With the patch, the query returns 7 using an Index Only Scan with Heap
Fetches: 0. make check (239 tests) and spgist_name_ops pass. The patch
is on top of v36.

I also checked whether the other built-in index AMs have the same
problem. This requires (1) an index-only scan needing only an INCLUDE
column, (2) a key whose original value cannot be reconstructed from its
stored type, and (3) an AM that puts that stored value into a tuple
described with the original key type. GiST can encounter the first two
conditions, but gistFetchTuple() sets the key column to NULL when it
cannot reconstruct the original key value. I used the same approach in
the attached 0001. I ran the same polygon/INCLUDE(id) query with GiST on
v36: it returned 7 with Heap Fetches: 0. B-tree returns its stored index
tuple without this reconstruction. Hash, GIN and BRIN have no
amcanreturn callback, so their scans cannot take this index-only path.
Another AM would be affected if it met all three conditions.

Regards,
Rui