[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: Open Bugs in 3.2
From:       Nicolas Goutte <nicolasg () snafu ! de>
Date:       2004-01-12 17:07:13
Message-ID: 200401121807.13188.nicolasg () snafu ! de
[Download RAW message or body]

On Monday 12 January 2004 17:56, Rüdiger Knörig wrote:
> That can't be a FPU-related error since
> 49.84 + 33.52 equals 4.984e1+3.352e1. Since the exponents doesn't differ

That is still in decimal. The FPU calculates in binary (as far as I know, 
unlike hardware calculators, even give-away ones.)

> the FPU doen't need to perform any shifting operation to "get the dots in a
> column" thus additions may only get inaccurate by overflows - but this

You are missing: a decimal to binary conversion, calculating in binary, 
retransforming in decimal.

> addition example won't even result in a denormalized floating-point number.

However probably in binary.

>
> Thus there must be something wrong with kcalc.

It seems that it was found. At one place it used double instead of long 
double, therefore the error after the 15th digit. With long double, you have 
a few bits more, therefore a few digits precision more.

Have a nice day!

 
>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<

[prev in list] [next in list] [prev in thread] [next in thread]