5-Apr-87 21:09:42-PDT,11286;000000000000 Mail-From: NEUMANN created at 5-Apr-87 21:08:55 Date: Sun 5 Apr 87 21:08:55-PDT From: RISKS FORUM (Peter G. Neumann -- Coordinator) Subject: RISKS DIGEST 4.71 Sender: NEUMANN@CSL.SRI.COM To: RISKS-LIST@CSL.SRI.COM RISKS-LIST: RISKS-FORUM Digest Sunday, 5 April 1987 Volume 4 : Issue 71 FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: Re: A real eye-catching headline -- nuclear safety (Jerry Saltzer, Peter G. Neumann, Henry Spencer) A non-fail-safe ATM failure (Don Chiasson) Fumes from computers and other electronic appliances (Richard Thomsen) Open University Fire (Lindsay F. Marshall) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, nonrepetitious. Diversity is welcome. (Contributions to RISKS@CSL.SRI.COM, Requests to RISKS-Request@CSL.SRI.COM) (Back issues Vol i Issue j available in CSL.SRI.COM:RISKS-i.j. MAXj: Summary Contents Vol 1: RISKS-1.46; Vol 2: RISKS-2.57; Vol 3: RISKS-3.92.) ---------------------------------------------------------------------- Date: Fri, 3 Apr 87 17:49:56 EST To: RISKS FORUM (Peter G. Neumann -- Coordinator) Subject: Re: A real eye-catching headline [David Chase, RISKS-4.70] From: Jerome H. Saltzer > "Inherently safe nuclear reactors" > [Add to the oxymoron list. PGN] Before assuming nonsense, one might try reading the article under the headline. It explores a series of design approaches with the common theme that safety mechanisms should be driven by simple, passive, inexorable laws of physics rather than being complex gadgetry in themselves. (For example, place the entire reactor system under water so that faults that would usually produce loss-of-coolant failures tend to instead produce too-much-coolant failures.) Whether or not the specific technical ideas are competent I can't judge, but the notion of designing safety measures that are simple and inevitable seems something that people concerned with computer RISKS should want to ponder rather than laugh at. Jerry ------------------------------ Date: Fri 3 Apr 87 16:55:55-PST From: Peter G. Neumann [Edited] Subject: Re: A real eye-catching headline To: Saltzer@ATHENA.MIT.EDU cc: RISKS@CSL.SRI.COM Yes, indeed, I certainly agree that one should understand something well enough before making light of it. In fact, the IEEE Spectrum article is quite significant. Use of inexorable laws of physics is a marvelous idea -- if those laws are in fact complete, correctly understood, immutable, and nonbypassable... The principle is excellent in the small. The practice may not be so easy to guarantee in the large. [See also Henry Spencer's message below.] I would like to add something that addresses not the inexorability, but rather the limitations of the environment in the case of large-scale nuclear power (to which this technique has not yet been applied): 1. People are not infallible, and are certainly not "inherently safe". Incompent or careless people might make an "inherently safe" nuclear reactor ACTUALLY UNSAFE. PBS' All Things Considered on 3 Apr 87 concluded that BAD MANAGEMENT was probably the biggest source of problems. Bad management is quite capable of rendering a system inherently unsafe, e.g., as a result of unwise cost-saving measures. (Philadelphia's Peach Bottom plant was just closed by the NRC; a surprise visit found the operators sleeping on the night shift. The PBS program also noted a safety system installed backwards.) 2. The inexorable laws (in the small) may be circumvented under actual environmental conditions, i.e., to the system in-the-large -- via accidents, sabotage, earthquakes, carelessness, and improper maintenance, as well as bad management and other human behavior noted above. [ 3. Despite claims to the contrary, nuclear waste disposal appears to be at least LONG-TERM RISKY, and may prove to be INHERENTLY UNSAFE. There appear to be no really appealing solutions in the long run, but that argument is beyond the scope of RISKS. I toss it in simply to illustrate the holistic nature of the problem and the nonholistic nature of the assumptions of infallibility. ] It does seem that assumptions are being made about the INFALLIBILITY of the technology. I quote from the Spectrum article: "If a major system fails, for example, the core is flooded automatically with coolant that flows under immutable laws of gravity and thermohydraulics, not under propulsion by mechanical pumps and electromagnetic actuators." If applied to nuclear power, does this ignore all sorts of fallibilities? Are there not still combinations of mechanisms and components that might fail, e.g., if the coolant suddenly springs a major leak, or if during maintenance reliance on the "INHERENTLY SAFE" physical principles must temporarily be circumvented, or if people do not always behave reasonably, as assumed? The notion that it is possible to design something that is "100% reliable" UNDER ALL POSSIBLE CIRCUMSTANCES is clearly unrealistic. But, even "99.9999% reliable" is not very good if the .0001% case can be provoked accidentally or intentionally by a specific combination of plausible circumstances (whether anticipated or unanticipated). Of course, A VERY REAL RISK LIES IN BELIEVING IN THE INFALLIBILITY OF TECHNOLOGY. Nevertheless, the cited April 87 IEEE Spectrum article is worth reading, and Jerry's points are very well taken. For those technologies in which risks can be substantially reduced by using homeostatic processes, that should be encouraged. (Although nuclear power has not yet been so based, the article makes an important point that it should be!) A good example of homeostasis is the human body, which is basically self-regulating -- except that when it breaks down, all bets are off. (To the reader: I know that Jerry doesn't believe in the infallibility of technology. I am not trying to shoot a straw herring in the foot. This message is by way of further discussion.) Peter ------------------------------ Date: Sun, 5 Apr 87 16:46:06 pst From: pyramid!utzoo!henry@hplabs.HP.COM To: pyramid!hplabs!CSL.SRI.COM!RISKS@hplabs.HP.COM Subject: Re: A real eye-catching headline > IEEE Spectrum, April 1987: > "Inherently safe nuclear reactors" > [Add to the oxymoron list. PGN] Not so, actually. The things actually exist, and the term accurately describes them. You could take a sledgehammer to the controls and nothing much would happen. U of T has one. Apparently if you're the last one to use it Friday afternoon, you just lock the door behind you and leave it unattended for the weekend. Unlike power reactors, the design is inherently stable: an increase in temperature causes a decrease in reaction rate, so nothing you can do will make it overheat. Unfortunately, the design does not scale up well and hence isn't useful for power plants. Henry Spencer @ U of Toronto Zoology {allegra,ihnp4,decvax,pyramid}!utzoo!henry [The Spectrum article suggests that it COULD be useful for power plants... PGN] ------------------------------ Date: Thu 2 Apr 87 08:38:46-AST From: Don Chiasson Subject: A non-fail-safe ATM failure [Still one more interesting ATM saga!] To: neumann@CSL.SRI.COM ReSent-To: RISKS@CSL.SRI.COM I'd like to pass along one which happened to me and indicates the risk of interactions between computers and mechanical components in automated systems. A few months ago I used an ATM to pay a bill. At the end of the transaction, the machine said to: "REMOVE CARD TO QUIT OR PRESS OK TO CONTINUE". I was done, so I pulled out my card and started to leave. I took a few moments getting my credit card back in my wallet, then had one of those "What wasn't that??" feelings. The door on the ATM hadn't gone down. The mechanical switch which shows whether my card is or is not in the machine had stuck and the ATM was patiently waiting for my next transaction. I aborted it by pressing a "CANCEL" button. Had I not done that, anyone passing by (this machine was outdoors) could have pressed a few buttons and paid their own bills from my account or pulled out my daily cash limit. Lesson: verify that the machine is doing its thing. Don [Despite a RISKS moratorium on routine ATM stories, this one is worth including as an example of an uncompleted supposedly atomic transaction with nasty side-effects. Another example of inconsistency between the state of the software and the state of the hardware was the THERAC 25 therapeutic radiation device: you recall that the software thought it had switched the device to X-ray mode (1,000 rads), but the device was still in electron-beam mode (up to 25,000 rads) at the moment. (See RISKS-3.9.) PGN] ------------------------------ Date: Thu, 2 Apr 87 07:33:33 mst From: rgt@LANL.ARPA (Richard Thomsen) To: risks@csl.sri.com Subject: Fumes from computers and other electronic appliances Just as radon gas comes from cement walls and formaldehyde comes out of new housing walls, carpets, and some modular furniture, there are gases that are emitted from electronic appliances. I do not know what they are, but suspect they come from the plastic cases, and probably the circuit boards and capacitors. I know someone who is highly allergic to chemicals, and can smell these gases. They are more prone to be emitted when the computer is on, since it is warmer. [This subject needs some real expert contributions. PGN] ------------------------------ From: "Lindsay F. Marshall" Date: Sun, 5 Apr 87 11:29:51 gmt To: risks@csl.sri.com Subject: Open University Fire Recently there has been considerable publicity given to a fire that took place in a computer room at the Open University HQ. The fire destroyed a VAX and all its back-up tapes that were stored in the machine room. The interesting thing about this event was the various reports of the problems caused by the loss of the filestore back-up. Initial (non-trade) press reports talked about people losing 15-20 years of research, this was then whittled down to two to three years (in the trade papers) and eventually seems to have come down to a couple of months as most people had personal back-ups of critical data!! Lindsay F. Marshall, Computing Lab., U of Newcastle upon Tyne, Tyne & Wear, UK ------------------------------ End of RISKS-FORUM Digest ************************ -------