Software Quality Assurance

What is the process?

"Our" lifecycle may be:

Waterfall (linear-sequential), Spiral, Incremental,

But all include: Code Generation and Testing

What are our "goals" ? (What do we want from the

software development process?)

We want:

To be able to develop software quickly (calendar

months)

To minimize resource needs: (few(er) people,

for a shorter time - effort)

To create re-useable components (modules, designs,

even requirements)

To reduce costs of fixing defects in delivered software

???? Can we do this:

(For a 200 KLOC DP product)

CMM level 1 organization:

Duration 30 months (6.7 KLOC/Month)

Effort 594 Person Months

Total Cost $5,440,000

Duration 15 months (13 KLOC/Month)

Effort 79.5 Person Months

Total Cost $728,000

Duration 9 months (22 KLOC/Month)

Effort 16 Person Months

Total Cost $146,000

So we know how to do what we want (above)

(CMM)

Institute Basic Project Management (repeatable)

We define and document processes

Measure and manage quality and productivity

Optimize and continually improve processes

We have seen the costs of fixing defects increase

later in the development process

To minimize the costs, fix defects early. How?

Even if the costs are minimized, if the product is not

high "quality" it won't meet requirements

(Or won't be used, or won't sell)

What is quality?

Which is better:

Cars:

Yugo, (Trabi, Maruti, etc) OR

Cadillac (Saturn, M.Benz, Volvo, etc.)

Why?

Who makes good computers?

Who makes good electronics, cameras?

Who makes good software?

Why? How do you determine?

(Who makes bad software?)

How can we determine "quality"?

The (old?) American Car Companies method:

"We getting better all the time"

(Don't change anything…)

The Saturn (Japan, from older US) method:

Measure now (metrics)

Improve the metrics

Quality can be:

Speed

Capacity (sizes, etc.)

Number of defects

Kinds of defects ("cosmetic", or lose data)

V and V

Validation: Are we building the right product?

Verification: Are we building the product right?

(Boehm)

Quality control is: inspections, reviews, and tests

with feedback.

Quality Assurance

Auditing and reporting aspects of management.

Is quality free?

(There was a book by this name) NO.

What are the costs of quality?

Building in quality:

(Prevention)

Planning

Formal Technical Reviews

Training

Special software, equipment

Appraisal:

Inspection

Testing

Costs of failure (defect):

(In house, before shipping)

Rework

Repair

Analysis of failure

Missed deadlines

Low morale

Dismissing employees (firing, killing)

(In the field, after shipping)

Product Return

Canceled sales

Penalties

Help desk support

Warranties

SQA

Based on requirements

(What else?)

Standards and development steps measured

"Implicit requirements" - maintainability, useablity

In other industries:

Bell Labs in 1916, formal QA

Many other "test" groups

In software:

Was the responsibility of programmer

(small systems)

Now?

Independent test groups

SQA (needs to:)

Prepare SQA plan

Participates in development of software process

description for a project

Reviews SE for compliance

Audits work (for compliance)

Documents deviations

Records and Reports noncompliance

Software reviews:

Use other people (diverse group) to:

Suggest improvements

Eliminate unnecessary work

Make product: more manageable, predictable,

uniform

What are defects?

Errors, faults, bugs, bugettes

As defects occur in a phase, if they are not removed

in that phase (step), they are "amplified" (effort, cost)

in the next and latter phases (steps)

Formal Technical Reviews:

(walkthroughs, inspections, reviews)

Goal: Find errors, verify requirements, follow

standards, training

FTR:

3 to 5 people

peers, no managers

Advance preparation required, no more than

2 hours

Meeting less than 2 hours

FTR guidelines:

review product, not person

Set agenda, follow it

Limit debate and response

Don't solve problems

Take written notes

Limit participants and prepare in advance

Schedule resources, etc

Training for reviewers

SQA

(Use statistics)

Collect and categorize defect information

Trace underlying cause

Pareto principle (80 % defects in 20% causes -

Correct those

MTBF = MTTF + MTTR

Availability (per cent of time)

Safety and hazards: how dangerous

SQA plan: what documents?

ISO 9000 (ISO 9001, ISO 9000-3)

what must be done, not HOW