[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: [despammed] Re: Open Bugs in 3.2
From:       Esben Mose Hansen <esben () mosehansen ! dk>
Date:       2004-01-12 16:58:48
Message-ID: 200401121815.34308.esben () mosehansen ! dk
[Download RAW message or body]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 12 January 2004 00:32, Joachim Eibl wrote:

> Check this on your machine.
>   long double a=49.84;       // double constant
>   printf("%2.30llf\n",a);        // 49.840000000000003410605131648481
>   long double a1=49.84L;   // long double constant
>   printf("%2.30llf\n",a1);      // 49.840000000000000000138777878078
>
> This just illustrates how tricky it can be.
> BTW: There is no real general solution. You might just settle for a
> compromise like showing fewer digits. But it won't help you when you start
> subtracting almost equal numbers like: 1000.0000 - 1000.0001 etc.

The general solution is: don't convert to binary floating point. Keep it in 
decimal floating point (as vector<char> (decimals)+ long (decimal point) or 
whatever) and do the calculations there. Not something I would propose for 
KCalc at this stage, though. There would probably be (L)GPL libraries for 
this anyway.

Why not simply limit KCalc to e.g. 13 significant digits? That's what 
calculators used to do, and it works fine 99.999% of the time ;-)

- -- 
regards, Esben

Homepage: http://www.mosehansen.dk
Signature fingerprint at http://www.mosehansen.dk/about
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQFAAtY1rfnftt13wXIRAlmRAJ9u++Msy+iScvJBoPmdk7D68EhPmQCeMKZF
nrjpN6NY5t8KPqKgiPLktzk=
=AHZE
-----END PGP SIGNATURE-----

 
>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<

[prev in list] [next in list] [prev in thread] [next in thread]