Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error

From: Victor Petrovykh <victor(at)magic(dot)io>
To: andrew(at)tao11(dot)riddles(dot)org(dot)uk
Cc: tgl(at)sss(dot)pgh(dot)pa(dot)us, pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error
Date: 2018-11-24 01:16:23
Message-ID: CAJ-A5aNZjv+BSg08ZtMwsmiqucKcBeSirfYs16AnY5azD=ZHZw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

I'm sorry I'm kinda new here.

Am I missing something in thinking that the cast of a float to int should
produce an error if the sign of the original value doesn't match the sign
of the cast result? Presumably you have access to both values at some point
and checking sign bits for sameness should be simple.

Also, I would say that doing some float4 arithmetic and then casting into
int4 or int8 should be consistent, so if

SELECT (54610::float4 * 39324::float4)::int8;

gets me 2147483648, then

SELECT (54610::float4 * 39324::float4)::int4;

should be an error rather than the unintuitive -2147483648.

The scenario that I'm pointing at is basically first doing some float4
arithmetic, casting the result into an int4 and expecting that the result
(if it was without error) is actually about as close to the original float
value as float4 precision would normally allow.

On Fri, Nov 23, 2018 at 7:45 PM Andrew Gierth <andrew(at)tao11(dot)riddles(dot)org(dot)uk>
wrote:

> >>>>> "Tom" == Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:
>
> Tom> if (unlikely(num < (float4) INT_MIN || num >= (float4) INT_MAX ||
> isnan(num)))
>
> >> if (num < (float4)INT_MIN || num >= -(float4)INT_MIN || ...
>
> Tom> Meh. Seems to me that's relying on pretty much the same
> Tom> assumptions
>
> No, because we know that INT_MIN is always exactly representable as a
> float (whereas INT_MAX is not), and therefore the cast result will not
> depend on any rounding choices whether at compile or run time. Nor does
> it depend on knowing that float4 can't represent INT_MAX exactly.
>
> --
> Andrew (irc:RhodiumToad)
>

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Tom Lane 2018-11-24 01:42:36 Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error
Previous Message Andrew Gierth 2018-11-24 00:45:38 Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error