CSE 5324: Software Engineering I

(Analysis, Design, Creation)

Fall 1997

Software Testing

You can not prove that no defects are

present, only that there are defects.

What is the objective of testing?

Finding errors. (defects, bugs)

Guarantee quality

There are many methods for testing:

What do exams test?

Statistical sampling testing

One can test for "related" artifacts (markers)

Some chemical testing is done by testing for by-products

Mining tests for other products usually found with what

is being searched for.

How to find (test) suitability of novia?

(girlfriend, boyfriend, significant other, etc.)

Go to a restaurant.

Watch how waiter is treated.

That’s you, in a few months.

Testing is not so easy:

Specification for Sort:

Input specification:

p: array of n integers, n> 0

Output specification:

q: array of n integers such that:

q[ 0 ] <= q[ 1 ] <= … <= q[ n -1 ]

Implementation:

void Sort ( int p[ ], int q[ ] )

{

int I;

for ( I = 0, I<n, I++)

q[ I ] = 0;

}

Specification is wrong!

The elements of the array q are a permutation of

the elements of p, which are unchanged

Even when the specification is correct

Ask user for two numbers, add them together, print

results

How big are numbers? What base? Negative? Etc.

An example implementation:

Int A, B, Sum

Read A, B

Sum = A + B

Print Sum

Exhaustive Testing

All possible values for A and B

assuming a 64 bit CPU

A and B are 0 to +2^63-1 ( -1 to -2^63)

even at 1000000 tests per second,

this takes years.

What are Objectives of testing

Glen Myers

Testing is process of executing program with

intent of finding error

Good test case high probability of finding

as yet undiscovered error

Success is finding as yet undiscovered error

WE WANT TO FIND ERRORS (not avoid them)

Principles by Davis

Testing traceable to requirements

Planned long before testing starts

Start "in the small" then go larger

Not possible to exhaustive test

Effective - Third party tests

How testable is a program (system)?

Operability

The better it works, easier to test

Defects block testing

Defects slow process: reporting, analysis

Observability

Test what you see

For an input, distinct output

Can query state

Logging

Controllability

All outputs can be generated by inputs

All code can be tested by inputs

Simplicity

Small systems more testable than large

Functional simple

Modularity of code

Stability

Change disrupts testing

Change invalidates previous tests

What is a good test?

Kaner, et al

High probability of finding an error

Not redundant

Best of its type

Not too simple or complex

Why not just "prove" correctness?

The sad case of "correctness proofs"

The programmer should let the proof and the program

grow hand in hand.

Dijkstra

Naur (1969) published a paper on constructing and

proving a product correct.

The line editing problem:

Given text seperated by blanks or newline,

convert, line - by - line to:

line breaks only where blank or new line

line filled as far as possible

no line more than maxpos characters

Data:

This is an example

of some

lines of text that can be very very very long

Program was 25 lines of Algol

Leavenworth in 1970 found first line is

preceded by blank unless exactly maxpos

char long.

Test data would have shone this

London in 1971 found 3 more errors

London publihed corrected proof

Goodenough and Gerhart found 3 more

errors (testing would find )