CSE 5324: Software Engineering I

(Analysis, Design, Creation)

Fall 1997

Software Testing

(Part 2)

What is the objective of testing?

Finding errors. (defects, bugs)

Guarantee quality

How can one test?

Designing test cases can be as difficult (or more)

than implementation.

Many (most) products can be tested in two ways:

Black Box

when we know the function of the product

we can test each function

when this is software - this is testing at the

interface level.

(don't need to know internals)

White Box

when the internal structure is known

we can test that all parts are exercised

for software testing paths through logic

(based on internals)

White Box testing can be impractical:

Exhaustive testing

Both methods are important (but not

equally used)- different types of errors

are found

White Box ( or clear, transparent or glass box)

what type of errors are common?

logic errors occur more frequently in less

used paths (special cases are "special"

and more difficult)

we often miss-predict which paths are

commonly followed, program flow is

often counterintuitive

typographical errors are random

( I = j; ++k; ) syntax checks won't find

semantic errors (some languages better

than others - "C" vs ADA)

A Program to count the words in "standard input"

procedure CountWords;

var

nw: integer;

c: character;

inword: boolean;

begin

nw := 0;

inword := false;

while ( getc( c ) <> ENDFILE ) do

if ( c = BLANK ) or ( c = NEWLINE ) or (c = TAB )

then inword := false

else if ( not inword ) then begin

inword := true;

nw := nw + 1

end;

putdec( nw, 1 );

end;

1. procedure CountWords;

2. var

3. nw: integer;
4. c: character;

5. inword: boolean;

6. begin

7. nw := 0;

8. inword := false;

9. while ( getc( c ) <> ENDFILE ) do

10. if ( c = BLANK ) or ( c = NEWLINE ) or (c = TAB )

11. then inword := false

12. else if ( not inword ) then begin

13. inword := true;

14. nw := nw + 1

15. end;

16. putdec( nw, 1 );

17. end;

Some White Box testing methods

Basis Path Testing

A flow graph:

Nodes - 1, 2, 3

Edges - 1 ->2, 1 -> 3

Regions - Bounded Nodes by edges

Predicate Node - 1

We want to find a basis set of paths, a test

based on this basis set executes each statement

at least once

Cyclomatic Complexity

A metric that quantitatively measures

logical complexity

The number of independent paths in the basis

set

Paths:

path 1: 7,8 -> 9 -> 9a -> 10a -> 10b -> 10c -> 12 -> 9 ->

path 2: 7,8 -> 9 -> 15 -> 16

etc.

should not be combinations of other paths

Basis sets are not unique

If we force execution of each path in basis set

every program statement is executed at least once

How many paths are there?

Cyclomatic complexity

V(G) = E - N + 2 (Edges, Nodes)

or

Number of Regions

or

V(G) = P + 1 (Predicate Nodes)

V(G) is upper bound of independent paths

upper bound of number of tests to be designed

and executed

Deriving test cases:

Use design (or code), draw flow graph

Determine V(G)

Find basis set of linear independent paths

Prepare test cases to force execution of each

Can use "Graph Matrix"

NODE

1 2 3 4 5

1 1

2 1 1

3 1 1

4

5

If use 0, 1 the connection exists or not

may weight with probability

processing time, other resource use

Testing at control structure

Condition testing

Expression relational-operator Expression

Errors include: Rel-op error, arithmetic error, other

test each condition in program

by: branch testing

domain testing ( domain of boolean,

Data Flow Testing

Select Paths according to definition and use of

variables

Def - Use

DU chain:

Where variable is defined, first set, used,

Loop testing

Simple loop test must

skip the loop

make exactly one pass

two passes

m passes, m < limit

limit, limit+1, limit-1 passes

Nested Loops

Start at innermost loop

test holding outside loops at minimum val.

work outward

until finished

Spaghetti loops (unstructured)

Re-do them

Black Box Testing:

Sometimes called: Behavioral or partition testing

Find:

Wrong or missing functionality

Interface errors

Data Structure Errors

Initialization, Termination Errors

Meyer says:

Test cases should reduce need for additional

test cases, tell us about classes of errors,

not just single errors

Graph based

Graph of objects and relationships

Nodes, node weights, links, link weights

Beizer describes

Transaction flow modeling

Finite State modeling

Data Flow

Timing modeling

May or may not be symmetric

Test cases cover: Nodes, Links

Equivalence Partition

Input domain is divided into classes

Input in ranges: valid parts of domain and invalid

domain: 5..10 is valid, 0..4, 11..100 is invalid

Input generated into each equivalent partition

Boundary Value

Very Common

Find range, test just above and below

Comparison Test

Create several different implementations

Test and compare

What are interesting to test:

GUI

Client/Server

Distributed

Documentation, Help

Real Time

What does the "real world" do?

Probably the wrong thing.