Re: [PATCH] Normalize numeric/bigint scientific notation after rounding

From: Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
To: Jianqiao Han <hanjianqiao(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [PATCH] Normalize numeric/bigint scientific notation after rounding
Date: 2026-11-16 10:00:40
Message-ID: CAEZATCXRGF8hS6WNytDX7PF1b4Wyr4RMn3DK=L_T=4YXvw59Xw@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, 16 Nov 2026 at 00:02, Jianqiao Han <hanjianqiao(at)gmail(dot)com> wrote:
>
> On Sat, 14 Nov 2026 at 02:28, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >
> > Good point. The patch feels a little brute-force, but I don't see
> > a simple way to do better, and it's probably not worth a huge
> > amount of trouble to make this edge case faster.
> >
> > Is there any case in which we have the reverse problem, ie rounding
> > down causes the leading digit to become 0?
> >
> Thanks for the review. The patch handles the tmp_var calculated by
> following two function calls:
>
> power_ten_int(exponent, &tmp_var);
> // rscale is always >= 0
> // for a non-zero var: var / 10^exponent is in [1, 10), with round=true the
> // range is [1, 10] ie. 9.99 will be rounded to 10.0
> div_var(var, &tmp_var, &tmp_var, rscale, true, true);
>
> And the patch increases exponent only when tmp_var.digits[0] >= 10.
> So for a non-zero input the tmp_var.digits[0] within range [1, 10] will
> not be rounded to zero, and I cannot think any other type of reverse
> problems will occur here.

Agreed. If it doesn't round up, it truncates, which doesn't change the
leading digits.

The patch looks correct to me, though the same result can be achieved
without a second div_var() call, and I think it would be worth adding
Assert()s to confirm the current division result is as expected.

After the current call to div_var(), if the input is non-zero
(var->ndigits > 0), tmp_var should have a magnitude in the range [1,
10], with rscale digits after the decimal point, so one should be able
to Assert the following for tmp_var:

ndigits > 0
weight == 0
dscale == rscale
digits[0] >= 1
digits[0] <= 10

In addition, if digits[0] == 10, the magnitude of tmp_var should
exactly equal 10, and any trailing zeros should have been stripped
from the digit array, so one could further Assert that ndigits == 1 in
that case.

Having confirmed all that, to compensate for it having 1 significant
digit too many when digits[0] == 10, all you'd have to do is set
digits[0] to 1 and increase exponent by 1.

The argument for doing it that way isn't so much about performance,
but more about confirming/documenting the algorithm.

Regards,
Dean

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message shveta malik 2026-11-16 10:03:01 Re: Initial table sync copies the whole table when it is no longer published
Previous Message shveta malik 2026-11-16 09:44:35 Initial table sync copies the whole table when it is no longer published