From koffice-devel  Tue Aug 20 02:24:58 2002
From: John Dailey <dailey () vt ! edu>
Date: Tue, 20 Aug 2002 02:24:58 +0000
To: koffice-devel
Subject: Re: patch to fix bug #18083
X-MARC-Message: https://marc.info/?l=koffice-devel&m=102980988706662

Looks good to me as a workaround for this release, commit if you haven't.

The problem overall isn't a problem with libc or anything else -- it's simp=
ly=20
a limitation of using the machine 'double' to store numeric values.  These=
=20
numbers that round evenly to thousandths in base 10 do not round cleanly in=
=20
binary so 63.145 might be stored internally as 63.14500000003 while 64.145=
=20
might be stored internally as 64.1449999999999996.  This accounts for why o=
ne=20
rounds up and one rounds down.  This can't be overcome without using either=
=20
infinite precision number storage, or by storing in base 10.

I am hoping we can integrate some system that is already written and well=20
tested into KSpread 1.3.

=2DJohn

On Monday 19 August 2002 04:11 pm, Norbert Andres wrote:
> Works for me.
>
> On Monday 19 August 2002 18:31, Ariya Hidayat wrote:
> > The following patch fix bug #18083 (kspread rounding error).
> >
> > Problem: in most system (including mine), printf("%.2f",x) results in
> > "63.15" if x is 63.145 but "64.14" if x is 64.145. For the latter, it
> > should be "64.15" (I checked with MS Excel). So it's not KLocale's or
> > QString's fault. I don't know exactly what might cause this (broken
> > libc?)
> >
> > Solution: round the number before passing it to KLocale::formatNumber().
> > Hopefully it doesn't introduce noticable penalty.
> >
> > OK to commit ?
> >
> > (John: sorry if this duplicates your work/investigation).
>
> _______________________________________________
> koffice-devel mailing list
> koffice-devel@mail.kde.org
> http://mail.kde.org/mailman/listinfo/koffice-devel

_______________________________________________
koffice-devel mailing list
koffice-devel@mail.kde.org
http://mail.kde.org/mailman/listinfo/koffice-devel