[prev in list] [next in list] [prev in thread] [next in thread] 

List:       log4cxx-dev
Subject:    Re: Testing strategy (was Re: cvs commit: logging-log4cxx/tests/witness/ndc
From:       Kevin Tew <listman () ekstrom ! org>
Date:       2005-01-07 16:24:30
Message-ID: 41DEB7BE.2010103 () ekstrom ! org
[Download RAW message or body]

Curt Arnold wrote:

>
> On Jan 6, 2005, at 4:45 AM, Ceki Gülcü wrote:
>
>>
>> Curt,
>>
>> While you are importing unit tests over from log4j, you probably
>> became aware that the log4j tests based on the witness approach,
>> although very valuable, are fragile and somewhat hard to maintain. I
>> suspect that better and cheaper testing methodologies are out there.
>>
>> What do you think?
>>
>
> The "witness" tests are effectively integration tests.  They only work 
> if a whole lot of things are working properly and if they fail, the 
> cause of the failure may be difficult to isolate.  It is good to have 
> "integration" tests, but they are more fragile and don't effectively 
> test atypical conditions.
>
> My current approach to dealing with reported issues in log4cxx is to 
> check that all the relevant log4j tests have been migrated.  If that 
> doesn't reproduce the reported bug, then I'll write new unit tests for 
> log4cxx that demonstrate the bug and donate those back to log4j.
>
> If I'm substantially rewriting some aspect (like the date 
> formatting).  I'll likely start by writing a lot of tests for log4j to 
> really pin down what it does, then port the tests and implementation 
> to log4cxx.
>
> The coverage analysis reports can be very helpful in guiding test 
> writing.  I find it most effective to try to push classes that have 
> fairly good coverage to 100%, then work on the next tier and favor 
> writing tests for the core classes.
>
> log4j has enough eyeballs on it that bugs get identified and squashed 
> pretty quickly.  Having a good test suite is definitely a benefit to 
> the derived implementations.  In a perfect world, the log4cxx test 
> suite would have all the tests that log4j has.
>
> If anybody wants to contribute to log4cxx by migrating log4j tests, 
> your help would be appreciated.


[prev in list] [next in list] [prev in thread] [next in thread]