From koffice-devel Sun Feb 11 00:36:00 2007 From: Brad Hards Date: Sun, 11 Feb 2007 00:36:00 +0000 To: koffice-devel Subject: Notes on koffice unit tests Message-Id: <200702111136.04971.bradh () frogmouth ! net> X-MARC-Message: https://marc.info/?l=koffice-devel&m=117115416030257 MIME-Version: 1 Content-Type: multipart/mixed; boundary="--===============1963077785==" --===============1963077785== Content-Type: multipart/signed; boundary="nextPart1949950.g9Hp5HB8OP"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit --nextPart1949950.g9Hp5HB8OP Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline I've been adding unit tests for kspread as part of the openformula work=20 (mostly it shows that kspread is pretty good for functions that have=20 reasonable input values, but the answers for "garbage in" often vary from=20 what you might expect :-) Anyway, that isn't really the point of this email. As of a few minutes ago,= =20 when I run the "make test" target for koffice from trunk, I get this: $ make test Running tests... Start processing tests Test project /home/bradh/build/koffice-build 1/ 38 Testing libs-store-storage_test Passed 2/ 38 Testing libs-store-storage_test2 Passed 3/ 38 Testing libs-store-xmlwritertest Passed 4/ 38 Testing libs-store-koxmlreadertest Passed 5/ 38 Testing libs-store-kodomtest Passed 6/ 38 Testing kofficecore-kogenstylestest Passed 7/ 38 Testing kofficecore-kooasissettingstes Passed 8/ 38 Testing kofficecore-kooasisstoretest Passed 9/ 38 Testing TestAction ***Exception: SegFault 10/ 38 Testing TestDocumentLayout ***Exception: SegFault 11/ 38 Testing TestBorder Passed 12/ 38 Testing flake-TestPosition Passed 13/ 38 Testing flake-TestSelection Passed 14/ 38 Testing flake-TestPathShape Passed 15/ 38 Testing flake-TestPathTool Passed 16/ 38 Testing flake-TestControlPointMoveComm Passed 17/ 38 Testing flake-TestPointTypeCommand Passed 18/ 38 Testing flake-TestPointRemoveCommand Passed 19/ 38 Testing flake-TestRemoveSubpathCommand Passed 20/ 38 Testing flake-TestShapeAt Passed 21/ 38 Testing TestShapePainting ***Failed 22/ 38 Testing pigment-TestColorConversion Passed 23/ 38 Testing Formula ***Failed 24/ 38 Testing OpenFormula Passed 25/ 38 Testing R-Tree Passed 26/ 38 Testing Value Passed 27/ 38 Testing MathFunctions Passed 28/ 38 Testing FinancialFunctions ***Failed 29/ 38 Testing TrigFunctions ***Failed 30/ 38 Testing PointStorage Passed 31/ 38 Testing BitopsFunctions Passed 32/ 38 Testing InformationFunctions ***Failed 33/ 38 Testing EngineeringFunctions ***Failed 34/ 38 Testing LogicFunctions Passed 35/ 38 Testing kword/frame/TestBasicLayout Passed 36/ 38 Testing kword/frame/TestDocumentLayout***Exception: SegFault 37/ 38 Testing kword-TestPageManager Passed 38/ 38 Testing TestDocumentLayout ***Exception: SegFault 74% tests passed, 10 tests failed out of 38 The following tests FAILED: 9 - TestAction (SEGFAULT) 10 - TestDocumentLayout (SEGFAULT) 21 - TestShapePainting (Failed) 23 - Formula (Failed) 28 - FinancialFunctions (Failed) 29 - TrigFunctions (Failed) 32 - InformationFunctions (Failed) 33 - EngineeringFunctions (Failed) 36 - kword/frame/TestDocumentLayout (SEGFAULT) 38 - TestDocumentLayout (SEGFAULT) The first problem is that it is sometimes not too obvious where to find the= =20 test (see 36 and 38 - same executable name) - maybe some kind of consistent= =20 naming convention (e.g. name equals directory position with - or /=20 separators) might be useful. Might also be a bit prettier. A more serious problem is that some of the tests are failing or segfaulting= ,=20 and that leads to "yeah, that's always broken" expectations. Looking at each of the failing tests: 9 is segfaulting in kdelibs, in the threadweaver Job destructor. 10, 36 and 38 may have something wrong with the test setup - dies in=20 QCoreApplication. Or maybe it is in my setup? 21 is failing both tests (paintedCount is 1 greater than expected in both=20 cases for lines 28 and 79), but why this happens requires much more flake=20 knowledge than I have. 23 is failing for some reason I don't understand - kspread formula parser=20 looks to be at fault. 28 and 29 appear to be failing because of excess precision in the compariso= n=20 of floating point numbers. 32 is failing because VALUE() doesn't parse fractions yet. I'm waiting on s= ome=20 info from Stefan on how to proceed on this. 33 is failing because we don't have IMCONJUGATE() yet. Stefan working on th= is? Are these results consistent with what other people are seeing?=20 Are we comfortable with using less than 15 digits in the floating point=20 comparisons? Note than the openformula spec uses epsilon of 1e-6 for some=20 tests, but I couldn't find a definition of what the "=3D" operator should t= reat=20 as "close enough". Is anyone working on the other problems? Brad --nextPart1949950.g9Hp5HB8OP Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQBFzmT0GwwszQ/PZzgRAgTrAJ0T9EKdoNfen3Nc1STH/gQP+RscxACaAnY7 Wt2kNbL22RASIDf4yWcR908= =AjIc -----END PGP SIGNATURE----- --nextPart1949950.g9Hp5HB8OP-- --===============1963077785== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ koffice-devel mailing list koffice-devel@kde.org https://mail.kde.org/mailman/listinfo/koffice-devel --===============1963077785==--