Hi all,
I have a patch for this and `make check` passes (core + all contrib, plus
new btree_gist NaN cases covering the three scenarios in the report).
Posting here first in case someone is already further along — if not,
I'll send it to pgsql-hackers and register it in the open CF in a couple
of days.
The fix swaps the five gbt_float{4,8}{gt,ge,eq,le,lt} comparators and
gbt_float{4,8}key_cmp for the NaN-aware float{4,8}_* helpers and
float{4,8}_cmp_internal() from utils/float.h — same total order the
btree opclass already uses. No on-disk change.
Bill