[prev in list] [next in list] [prev in thread] [next in thread] 

List:       sqlite-users
Subject:    Re: [sqlite] MC/DC coverage explained wrong in the home page?
From:       Sami Liedes <sliedes () cc ! hut ! fi>
Date:       2011-09-24 1:58:35
Message-ID: 20110924015834.GO31808 () sli ! homeunix ! net
[Download RAW message or body]

On Fri, Sep 23, 2011 at 09:17:43PM -0400, Richard Hipp wrote:
> ----------------------
> 1. Structural coverage guidelines are:
>   a) Every statement in the program has been invoked at least once;
>   b) Every point of entry and exit in the program has been invoked at least
> once;
>   c) Every control statement (i.e., branchpoint) in the program has taken
> all possible outcomes (i.e., branches) at least once;
>   d) Every non-constant Boolean expression in the program has evaluated to
> both a True and a False result;
>   e) Every non-constant condition in a Boolean expression in the program has
> evaluated to both a True and a False result;
>   f) Every non-constant condition in a Boolean expression in the program has
> been shown to independently affect that expression's outcome.
> 2. Based upon these definitions:
>   • Statement Coverage requires (a) only
>   • DC requires (b, c, d)
>   • MC/DC requires (b, c, d, e, f)
> ----------------------
[...]
> My claim is that in a short-circuit language, guideline e implies guideline
> f.  In other words, if all boolean operators are short-circuited, then
> obtaining e automatically means that you also obtain f.

But certainly (e) alone (without (c)) cannot imply (f). A simple
counterexample:

   if (A || 1) ...

You can get (e) by giving test cases for A and !A, but most certainly
flipping A does not "independently affect the outcome" as required by
the plain reading of (f).

Furthermore, I thought I just disproved the very claim that (b,c,d,e)
implies (f), by giving a counterexample where (b,c,d,e) are satisfied
but some of the conditions (namely B, C and D below) are *not* shown
to independently affect the outcome even where they are evaluated. :-)

The counterexample is quoted below.

	Sami

> > Yes, that reasoning makes sense. But even allowing for that doesn't in
> > all cases satisfy the fourth MC/DC criterion. It does for the simple
> > (A && B) case, but consider a more complex expression,
> >
> > ((A && B) || (C && D)):
> >
> > Now the truth table (with "_" as possibly uncomputable) would be
> >
> >      A  B  C  D   branch taken
> > (1)   0  _  0  _   F
> > (2)   0  _  1  0   F
> > (3)   0  _  1  1   T
> > (4)   1  0  0  _   F
> > (5)   1  0  1  0   F
> > (6)   1  0  1  1   T
> > (7)   1  1  _  _   T
> >
> > So using your approach, that is the plain Condition/Decision Coverage,
> > I believe these test cases would suffice:
> >
> >      A  B  C  D   branch taken
> > (1)   0  _  0  _   F
> > (2)   0  _  1  0   F
> > (6)   1  0  1  1   T
> > (7)   1  1  _  _   T
> >
> > This satisfies all three plain C/DC criteria:
> >
> > (a) Every point of entry and exit in the program has been invoked at
> >    least once -- does not apply to this if statement
> >
> > (b) Every condition in a decision in the program has taken all
> >    possible outcomes at least once -- A is tested by (1,6), B by
> >    (6,7), C by (1,2), D by (2,6)
> >
> > (c) every decision in the program has taken all possible outcomes at
> >    least once -- false branch taken in (1), false branch in (6)
> >
> > But this is not sufficient for the fourth criterion of MC/DC (quoting
> > from Wikipedia):
> >
> > (d) Each condition has been shown to affect that decision outcome
> >     independently. A condition is shown to affect a decision's outcome
> >    independently by varying just that condition while holding fixed
> >    all other possible conditions.
> >
> > This criterion is satisfied for only for condition A (by 1;7), but not
> > for B (6;7 would if the branches taken were different), C (1;2 would
> > if the branches taken were different) or D.
> >
> > Note that the "affects outcome" requirement really forces us to
> > consider the branch taken alongside with the conditions taken.
> > Short-circuiting operators are really mostly an orthogonal concern to
> > this. You simply cannot do MC/DC analysis without considering the
> > branch taken in conjunction with the values taken by the conditions.
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[prev in list] [next in list] [prev in thread] [next in thread]