Case Materials

Case Navigation

Therac-25 Case

Teaching Intro

Socio-Technical Analysis

Ethical Analysis

Exercises

Supporting Documents

General

Teaching with Cases

Social Impact Analysis

Computer Ethics Curriculum

Curricula Index

Case Materials

Therac-25

Machado

Hughes Aircraft

Ethics in Computing Links

Contact Us

Exercises for Therac-25

Computer Control Choices Exercise

EXERCISE: Use the range of human-computer control possibilities (on p. 448in Leveson) to locate Therac-25 control levels. Recommend and argue for a change in level. What would be required to move a level up? Down?

Choosing the Level of Computer Control

In her book Safeware: System Safety and Computers, Nancy Leveson lists nine different levels of computer control (taken from Sheridan’s analysis):

1. The operator does everything.
2. The computer tells the operator the options available.
3. The computer tells the operator the options available and suggests one.
4. The computer suggests an action and implements it if asked.
5. The computer suggests an action, informs the operator, and implements the action if not stopped in time.
6. The computer selects and implements an action if not stopped in time and then informs the operator.
7. The computer selects and implements an action and tells the operator if asked.
8. The computer selects and implements an action and tells the operator if the designer decides the operator should be notified.
9. The computer selects and implements an action without any human involvement.

After students have explored the case, have them decide at what level the Therac-25 system is targeted. This may initially cause some confusion, since one way of looking at the system is to think that the operator tells the computer what to do and then the computer does it. Point out to them that this is true in the larger sense, but that the computer clearly has sensors and information available to it to allow it to give error messages. What do we know about the level in this control hierarchy at which those error messages are resolved?

What levels of computer control is the system using when:

• an error message is given (e.g. Malfunction 54), but the system allows the operator to press a "proceed" key to retry the treatment.
• vs. (as required by the FDA) the treatment is suspended after any error and all treatment data must be typed in over again
• or, when the operator is required to "visually check the settings" on the treatment machine
• vs. when the machine sets itself up based on the treatment data entered and then proceeds with the treatment

Once you have established levels of computer control the machine is using, ask for suggestions about how one might increase the amount of computer control. What safety issue does this bring up?

One of the best ways to analyze the effects of changes in computer control is to have already completed the basic steps in the case analysis (determining stakeholders, duties and rights, opportunities and vulnerabilities).

References:
Leveson, N. G. (1995). Safeware: System safety and computers. New York: Addison Wesley.

Sheridan, T.B. (1989). Trustworthiness of command and control systems. In J. Ranta, (ed.) Analysis, Design, and Evaluation of Man-Machine Systems, (p. 427-431). New York: Pergamon Press.