[prev in list] [next in list] [prev in thread] [next in thread]
List: sqlite-users
Subject: Re: [sqlite] MC/DC coverage explained wrong in the home page?
From: Pavel Ivanov <paivanof () gmail ! com>
Date: 2011-09-24 2:26:43
Message-ID: CAG1a4rs0jAK1okmSwzBzBMO7a-wx57dLGsbGaPyT+tY4qMZphQ () mail ! gmail ! com
[Download RAW message or body]
> if (A || 1) ...
>
> You can get (e) by giving test cases for A and !A, but most certainly
> flipping A does not "independently affect the outcome" as required by
> the plain reading of (f).
I'm pretty sure that the latest versions of modern compilers will
optimize the above if statement to the following:
A;
// everything inside if
They won't even check the outcome of A and even won't calculate any
part of A that has no side effects (and it's known at compile time).
So in an object file A is not a condition and can hardly be called a
boolean expression. Does (d), (e) and (f) even apply to it?
Pavel
On Fri, Sep 23, 2011 at 9:58 PM, Sami Liedes <sliedes@cc.hut.fi> wrote:
> On Fri, Sep 23, 2011 at 09:17:43PM -0400, Richard Hipp wrote:
>> ----------------------
>> 1. Structural coverage guidelines are:
>> a) Every statement in the program has been invoked at least once;
>> b) Every point of entry and exit in the program has been invoked at least
>> once;
>> c) Every control statement (i.e., branchpoint) in the program has taken
>> all possible outcomes (i.e., branches) at least once;
>> d) Every non-constant Boolean expression in the program has evaluated to
>> both a True and a False result;
>> e) Every non-constant condition in a Boolean expression in the program has
>> evaluated to both a True and a False result;
>> f) Every non-constant condition in a Boolean expression in the program has
>> been shown to independently affect that expression's outcome.
>> 2. Based upon these definitions:
>> • Statement Coverage requires (a) only
>> • DC requires (b, c, d)
>> • MC/DC requires (b, c, d, e, f)
>> ----------------------
> [...]
>> My claim is that in a short-circuit language, guideline e implies guideline
>> f. In other words, if all boolean operators are short-circuited, then
>> obtaining e automatically means that you also obtain f.
>
> But certainly (e) alone (without (c)) cannot imply (f). A simple
> counterexample:
>
> if (A || 1) ...
>
> You can get (e) by giving test cases for A and !A, but most certainly
> flipping A does not "independently affect the outcome" as required by
> the plain reading of (f).
>
> Furthermore, I thought I just disproved the very claim that (b,c,d,e)
> implies (f), by giving a counterexample where (b,c,d,e) are satisfied
> but some of the conditions (namely B, C and D below) are *not* shown
> to independently affect the outcome even where they are evaluated. :-)
>
> The counterexample is quoted below.
>
> Sami
>
>> > Yes, that reasoning makes sense. But even allowing for that doesn't in
>> > all cases satisfy the fourth MC/DC criterion. It does for the simple
>> > (A && B) case, but consider a more complex expression,
>> >
>> > ((A && B) || (C && D)):
>> >
>> > Now the truth table (with "_" as possibly uncomputable) would be
>> >
>> > A B C D branch taken
>> > (1) 0 _ 0 _ F
>> > (2) 0 _ 1 0 F
>> > (3) 0 _ 1 1 T
>> > (4) 1 0 0 _ F
>> > (5) 1 0 1 0 F
>> > (6) 1 0 1 1 T
>> > (7) 1 1 _ _ T
>> >
>> > So using your approach, that is the plain Condition/Decision Coverage,
>> > I believe these test cases would suffice:
>> >
>> > A B C D branch taken
>> > (1) 0 _ 0 _ F
>> > (2) 0 _ 1 0 F
>> > (6) 1 0 1 1 T
>> > (7) 1 1 _ _ T
>> >
>> > This satisfies all three plain C/DC criteria:
>> >
>> > (a) Every point of entry and exit in the program has been invoked at
>> > least once -- does not apply to this if statement
>> >
>> > (b) Every condition in a decision in the program has taken all
>> > possible outcomes at least once -- A is tested by (1,6), B by
>> > (6,7), C by (1,2), D by (2,6)
>> >
>> > (c) every decision in the program has taken all possible outcomes at
>> > least once -- false branch taken in (1), false branch in (6)
>> >
>> > But this is not sufficient for the fourth criterion of MC/DC (quoting
>> > from Wikipedia):
>> >
>> > (d) Each condition has been shown to affect that decision outcome
>> > independently. A condition is shown to affect a decision’s outcome
>> > independently by varying just that condition while holding fixed
>> > all other possible conditions.
>> >
>> > This criterion is satisfied for only for condition A (by 1;7), but not
>> > for B (6;7 would if the branches taken were different), C (1;2 would
>> > if the branches taken were different) or D.
>> >
>> > Note that the "affects outcome" requirement really forces us to
>> > consider the branch taken alongside with the conditions taken.
>> > Short-circuiting operators are really mostly an orthogonal concern to
>> > this. You simply cannot do MC/DC analysis without considering the
>> > branch taken in conjunction with the values taken by the conditions.
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic