[prev in list] [next in list] [prev in thread] [next in thread] 

List:       koffice-devel
Subject:    Notes on koffice unit tests
From:       Brad Hards <bradh () frogmouth ! net>
Date:       2007-02-11 0:36:00
Message-ID: 200702111136.04971.bradh () frogmouth ! net
[Download RAW message or body]

[Attachment #2 (multipart/signed)]


I've been adding unit tests for kspread as part of the openformula work 
(mostly it shows that kspread is pretty good for functions that have 
reasonable input values, but the answers for "garbage in" often vary from 
what you might expect :-)

Anyway, that isn't really the point of this email. As of a few minutes ago, 
when I run the "make test" target for koffice from trunk, I get this:
$ make test
Running tests...
Start processing tests
Test project /home/bradh/build/koffice-build
  1/ 38 Testing libs-store-storage_test          Passed
  2/ 38 Testing libs-store-storage_test2         Passed
  3/ 38 Testing libs-store-xmlwritertest         Passed
  4/ 38 Testing libs-store-koxmlreadertest       Passed
  5/ 38 Testing libs-store-kodomtest             Passed
  6/ 38 Testing kofficecore-kogenstylestest      Passed
  7/ 38 Testing kofficecore-kooasissettingstes   Passed
  8/ 38 Testing kofficecore-kooasisstoretest     Passed
  9/ 38 Testing TestAction                    ***Exception: SegFault
 10/ 38 Testing TestDocumentLayout            ***Exception: SegFault
 11/ 38 Testing TestBorder                       Passed
 12/ 38 Testing flake-TestPosition               Passed
 13/ 38 Testing flake-TestSelection              Passed
 14/ 38 Testing flake-TestPathShape              Passed
 15/ 38 Testing flake-TestPathTool               Passed
 16/ 38 Testing flake-TestControlPointMoveComm   Passed
 17/ 38 Testing flake-TestPointTypeCommand       Passed
 18/ 38 Testing flake-TestPointRemoveCommand     Passed
 19/ 38 Testing flake-TestRemoveSubpathCommand   Passed
 20/ 38 Testing flake-TestShapeAt                Passed
 21/ 38 Testing TestShapePainting             ***Failed
 22/ 38 Testing pigment-TestColorConversion      Passed
 23/ 38 Testing Formula                       ***Failed
 24/ 38 Testing OpenFormula                      Passed
 25/ 38 Testing R-Tree                           Passed
 26/ 38 Testing Value                            Passed
 27/ 38 Testing MathFunctions                    Passed
 28/ 38 Testing FinancialFunctions            ***Failed
 29/ 38 Testing TrigFunctions                 ***Failed
 30/ 38 Testing PointStorage                     Passed
 31/ 38 Testing BitopsFunctions                  Passed
 32/ 38 Testing InformationFunctions          ***Failed
 33/ 38 Testing EngineeringFunctions          ***Failed
 34/ 38 Testing LogicFunctions                   Passed
 35/ 38 Testing kword/frame/TestBasicLayout      Passed
 36/ 38 Testing kword/frame/TestDocumentLayout***Exception: SegFault
 37/ 38 Testing kword-TestPageManager            Passed
 38/ 38 Testing TestDocumentLayout            ***Exception: SegFault

74% tests passed, 10 tests failed out of 38

The following tests FAILED:
          9 - TestAction (SEGFAULT)
         10 - TestDocumentLayout (SEGFAULT)
         21 - TestShapePainting (Failed)
         23 - Formula (Failed)
         28 - FinancialFunctions (Failed)
         29 - TrigFunctions (Failed)
         32 - InformationFunctions (Failed)
         33 - EngineeringFunctions (Failed)
         36 - kword/frame/TestDocumentLayout (SEGFAULT)
         38 - TestDocumentLayout (SEGFAULT)

The first problem is that it is sometimes not too obvious where to find the 
test (see 36 and 38 - same executable name) - maybe some kind of consistent 
naming convention (e.g. name equals directory position with - or / 
separators) might be useful. Might also be a bit prettier.

A more serious problem is that some of the tests are failing or segfaulting, 
and that leads to "yeah, that's always broken" expectations.

Looking at each of the failing tests:
9 is segfaulting in kdelibs, in the threadweaver Job destructor.
10, 36 and 38 may have something wrong with the test setup - dies in 
QCoreApplication. Or maybe it is in my setup?
21 is failing both tests (paintedCount is 1 greater than expected in both 
cases for lines 28 and 79), but why this happens requires much more flake 
knowledge than I have.
23 is failing for some reason I don't understand - kspread formula parser 
looks to be at fault.
28 and 29 appear to be failing because of excess precision in the comparison 
of floating point numbers.
32 is failing because VALUE() doesn't parse fractions yet. I'm waiting on some 
info from Stefan on how to proceed on this.
33 is failing because we don't have IMCONJUGATE() yet. Stefan working on this?

Are these results consistent with what other people are seeing? 

Are we comfortable with using less than 15 digits in the floating point 
comparisons? Note than the openformula spec uses epsilon of 1e-6 for some 
tests, but I couldn't find a definition of what the "=" operator should treat 
as "close enough".

Is anyone working on the other problems?

Brad

[Attachment #5 (application/pgp-signature)]

_______________________________________________
koffice-devel mailing list
koffice-devel@kde.org
https://mail.kde.org/mailman/listinfo/koffice-devel


[prev in list] [next in list] [prev in thread] [next in thread]