btree_gist: Fix NaN handling in float4 and float8 opclasses

Hi,

A GiST index on a float4/float8 column (via btree_gist) can silently
omit rows containing NaN, returning different results from a sequential
scan:

CREATE EXTENSION btree_gist;
CREATE TABLE t (a float8);
INSERT INTO t SELECT 1 FROM generate_series(1, 1000);
INSERT INTO t SELECT 'NaN' FROM generate_series(1, 1000);
CREATE INDEX ON t USING gist (a);

SET enable_seqscan = on;
SELECT count(*) FROM t WHERE a = 'NaN'; -- 1000
SET enable_seqscan = off;
SELECT count(*) FROM t WHERE a = 'NaN'; -- 0 (wrong)

"a >= 'NaN'" and "a > 'Infinity'" are affected the same way; float4
behaves identically.

The float4/float8 opclasses compare key values with the plain C
operators (>, >=, ==, <=, <), which follow IEEE 754 semantics where
every comparison with NaN is false, including NaN = NaN. PostgreSQL's
float types instead define a total order in which NaN equals itself and
sorts after every non-NaN value (float8_cmp_internal/float4_cmp_internal),
and that's what sequential scans and the in-core float B-tree opclasses
use. Because the consistent and union support functions used the raw
operators, union never widened an internal key to cover a NaN child and
consistent then pruned that subtree; since these consistent functions
report exact results (recheck = false), the wrong answer was returned
uncorrected.

The attached patch routes the gbt_float{4,8} comparators and key_cmp
through float8_cmp_internal/float4_cmp_internal. Everything else in the
opclass (union, picksplit, same, the consistent strategies) dispatches
through those same comparator callbacks, so this fixes the build and
scan paths together. It is the same mechanism commit 50354637694b used
when it made the in-core geometric GiST opclasses NaN-aware -- there the
symptom was a build-time problem rather than a silent wrong answer, but
the fix is identical in spirit (compare through float8_cmp_internal) --
and it matches the sortsupport comparators already present in these
files.

Regards,
Ewan Young