General
Teaching Tools
Case Materials
A Fixation on the Technical Failures (and Fixes) in the System
This pitfall is yet another version of simplistic thinking about the causes of accidents. Certainly technical failures in the system should be fixed. But a too narrow focus on these can lead one to think that simply fixing each one as it occurs is the appropriate action. For instance, one might think that the child safety problem with early refrigerators was that the latch could not be opened from the inside if a child was trapped inside. A narrow focus on this might lead one to add a "child detector" to the refrigerator that opens the latch when this troublesome condition occurs. This now leaves us with an additionally complicated system with more points of failure. Rethinking the design of the product allows one to see that one does not need latches to close refrigerator doors--magnets will do fine.
In a similar manner, we have several examples in the Therac-25 case of fixation on technical failures. The initial response of AECL involved pinpointing a microswitch failure as the problem. The claim that the technical fix to this produced a five order of magnitude increase in safety suggests that AECL felt this fix was the single solution. It took more than a year of negotiation with the FDA to get a plan from AECL that involved more systemic redesign issues.
One might ask the question: "Why have a dual mode medical linear accelerator in the first place?" At least one of the kinds of accidents from the Therac-25 would have been completely impossible if the machine used only a single mode. There are, of course, tradeoffs in this design decision. But negotiating design tradeoffs is standard fare for software engineers.