7-Feb-86 17:20:03-PST,13765;000000000000 Mail-From: NEUMANN created at 7-Feb-86 17:16:32 Date: Fri 7 Feb 86 17:16:32-PST From: RISKS FORUM (Peter G. Neumann, Coordinator) Subject: RISKS-2.8 Sender: NEUMANN@SRI-CSL.ARPA To: RISKS-LIST@SRI-CSL.ARPA RISKS-LIST: RISKS-FORUM Digest, Friday, 7 Feb 1986 Volume 2 : Issue 8 FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: Expert systems and shuttles (Michael Brown, Dave Platt) Plutonium (Martin J. Moore) Earthquake Monitoring Systems (Mike Raugh via Matt Bishop, Hal Murray, Eugene Miya) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, nonrepetitious. Diversity is welcome. (Contributions to RISKS@SRI-CSL.ARPA, Requests to RISKS-Request@SRI-CSL.ARPA.) (Back issues Vol i Issue j stored in SRI-CSL:RISKS-i.j. Vol 1: MAXj=45) ---------------------------------------------------------------------- Date: Fri, 7 Feb 86 09:17:13 est From: mlbrown@nswc-wo.ARPA To: neumann@sri-csl.arpa Subject: Expert systems and shuttles In Risks 2.7, J. Giles speculates: >Has expert system technology been thought of as a fix for this problem? >... a really fast computer ... could monitor all those inputs which aren't >under the direction of human flight controllers... >Are expert systems yet advanced enough to make this worthwhile? Unfortunately, expert systems developed to handle such an occurrence would have to be based on a foreknowledge of the relationship of the various anomalies that occurred in the shuttle disaster. I seriously doubt that most competent systems safety engineers could have predicted the occurrence even with a full knowledge of the anomalies that occurred. Development of such an expert system would likely have to be based on that type of knowledge. However, expert systems aside, I am amazed that the NASA systems safety people would permit a multiple section rocket motor to be manufactured at one location and assembled at another. Misfortune has shown us in the past that these composite structure solid rockets have some very unique and undesirable properties. It will be interesting to see exactly where the failure occurred in the shuttle's SRB if in fact it did fail. If the failure occurred at some location other than the suspect joint, chalk another one up to experience. Michael Brown ------------------------------ Date: Fri, 07 Feb 86 10:11 PST From: Dave Platt To: Risks Subject: Expert systems to detect shuttle failure References: Jim Giles' comments in Risks 2.7 Well, it's certainly possible to set up some sort of expert system that would monitor incoming telemetry and issue warnings in case of possibly-dangerous combinations of unusual conditions. However, I can see a couple of difficulties involved here: 1- There are some conflicts re the amount of data that you want to feed into the expert-system tool. Certainly, the more data that is available (from many different classes of sensors), the smaller the chance that the tool won't have the information needed to detect the problem. [For example, Challenger was equipped with far fewer sensors on the SRB than was Columbia during its tryoug flights]. But... as you increase the number of individual sensors, and the amount of data (# of different classes of data, especially), you necessarily increase the number of rules in the system, and the amount of crunchpower necessary to step through the rules and determine whether any conclusions need to be brought to the attention of the controllers. It doesn't do you much good to receive a warning saying "Engine failure is probable, based on conditions xxx and yyy" if you don't get the warning in time to do anything about it. In my [very limited] experience, very few if any existing expert systems are capable of handling large amounts of real-time data; the ones that I've seen tend to be somewhat sluggish. I don't doubt that it would be possible to build special-purpose hardware that would support such a system... but I don't believe it has been done yet. 2- As I understand them, expert systems are designed to reproduce (or mimic) the sort of what-if and maybe-then decision sequences that an expert would go through when analyzing a particular sort of problem. They work by encoding (in explicit form) the steps and conclusion that an expert would use. A large part of the work involved in developing an expert system is sitting down with the expert(s), and assisting them in encoding their (often implicit and unspoken) rules into rigorous form. All well and good... BUT... the expert system's "expertise" is entirely limited by the completeness of the rules that are used to construct it. One cannot assume that an expert system will be able to detect or diagnose a situation that has never been encountered before... it may do so, if the rules were complete enough and if the situation is similar to one that has occurred before, but you don't want to bet your life on it! Only the simplest expert systems can ever be considered to be "complete". When solving a complex, real-world problem (such as "Is the shuttle's current behavior normal?"), the best that you can expect is that some useful fraction of all possible situations will be analyzed in a meaningful fashion. Expert systems tend to grow and evolve as they are used... just as a human expert's capabilities do... and both humans and expert systems will tend to misdiagnose situations that fall outside of their current knowledge base. 3- Even if an expert system reacts quickly and accurately enough to give a meaningful warning ("SRBs leaking, ET overheating, explosion imminent"), you're still faced with: [A] Human reaction time (controller and pilot); [B] taking the necessary immediate action (split the SRBs from the ET and/or split the orbiter away from the ET); and [C] surviving (getting far enough away from the ET before it goes *BLOOIE*, and then completing a very difficult dead-stick turnaround and landing, or a tough water ditching). In the case of the Challenger explosion, it looks as if all three of these factors were dead-set against the crew... there was very little time to react, no way to get away, and a water ditching would probably have killed many of the crew. I imagine that you could certainly build an expert system that would be capable of reading the shuttle's telemetry, and warning of most conditions THAT THE DESIGNERS OF THE SYSTEM HAD TAKEN INTO ACCOUNT! The real problem lie, of course, in detecting conditions that no one had expected would occur... if the system has no rules that would lead to a conclusion such as "The SRB segment ring seals are leaking", then the system will never report such a condition. At best, some other warning will be reported ("Asymmetric thrust from SRBs exceeding 2%"); at worst, no warning will be received, or the system will issue warnings unnecessarily ("Heavy engine vibration"). ------------------------------ Received: from eglin-vax.ARPA ... Fri 7 Feb 86 12:20:06-PST Date: 0 0 00:00:00 CDT From: "MARTIN J. MOORE" Subject: Plutonium To: "risks" I don't think the worries about plutonium should be dismissed out of hand. It is my understanding that the lethality of plutonium is due to its extreme toxicity, as opposed to its radioactivity. Comments from a knowledgeable chemist are eagerly solicited. ------------------------------ Date: 7 Feb 1986 1502-PST (Friday) From: Matt Bishop Organization: Research Institute for Advanced Computer Science Address: Mail Stop 230-5, NASA Ames Research Center, Moffett Field, CA 94035 Phone: (415) 694-6363 [main office], (415) 694-6921 [my office] To: risks@sri-csl Subject: Re: Earthquake Monitoring Systems I took the liberty of forwarding Gary Leaven's question on earthquake monitoring systems (ie, are they designed to function during a major earthquake?) to Mike Raugh, the author of the CACM article which prompted the question. Here's his reply: --------------------------------- Matt, The question you forwarded to me is a good one: Are the seismic instruments used in Calnet and the Southern California Array built to withstand the shaking of a major earthquake? The answer is Yes and No, but it doesn't matter! Even if a local subset of instruments (or the telemetry system serving that subset) is knocked out by a major quake, more distant instruments will pick up signals from the quake that will be adequate for locating, timing and calculating the earthquake "mechanism", i.e. direction of first motion, plane of rupture, magnitude. The purpose of the two arrays is to monitor earthquake activity throughout California, so you can see that the entire combined two arrays will almost certainly not be totally incapacited by a major quake, hence they will continue to monitor activity (even distant activity) successfully. That being said, it should also be mentioned that seismologists are very interested in the fine-grained signals that are obtainable only at close range to a major earthquake (seismic waves that have traveled teleseismic distance through the earth lose much of the higher frequency energy). Such close-in data from large earthquakes can only be obtained from special "strong motion" instruments: this type of instrument furnished the data for Archuleta's study of the Imperial Valley quake discussed in my paper. Strong motion instruments are much more difficult to make, for all the obvious reasons, and are expensive compared to the ones that comprise the two arrays mentioned above. The problem of designing sophisticated modern microcomputer based instruments that have sufficient sensitivity and dynamic range and are robust in the presence of violent shaking is a big one. Especially when you consider that such instruments must have local storage and power supplies to back up data collection in the event of telemetry break-downs. I can think of two groups at the USGS in Menlo Park that are working on systems of this kind. The first is lead by Roger Borchardt (his GEOS project was mentioned in my article). Another is being conducted by Larry Baker, Joe Fletcher, and Paul Spudich, who are developing a down-hole three-dimensional mesh of instruments for observation of the detailed progression of faulting expected to occur in the officially USGS-predicted earthquake at Parkfield. In other words, new designs for such instruments are on the frontier of research and development at the USGS. Very likely other work of similar import is taking place elsewhere. I hope this answers your question. Mike ------------------------------ Date: Fri, 7 Feb 86 03:13:43 PST From: Murray.pa@Xerox.COM Subject: Earthquake Monitoring Systems To: RISKS FORUM (Peter G. Neumann, Coordinator) Neither of these stories involves mainline computer risks, but they might contribute some insight. I got this story from a friend doing earthquake research for the USGS. I think it was '71 when a bigish quake near LA collapsed a VA hospital and a half constructed bridge. That generated a lot of interest in the way buildings (and bridges) react to quakes. Nobody really knew how much stress is present on various structural parts of a building. In response, many strain recording gizmos were installed in many large buildings. Time passed, and everybody went back to their normal work. After several years, another bigish quake came along, and somebody remembered all those installed instruments. So they went out to collected them. Most of them had died. I don't remember any numbers, but I was left with the feeling that everybody was discouraged that they didn't get much interesting data. Another friend worked on LASA (Large Area Seismic Array?). It was one of the early seismic arrays with hundreds of sensors scattered over eastern Montana. I think it was primarily part of the bomb test detection program. With that many sensors and that much wire and electronics to collect all the data, a few sensors were always off the air. They discovered that they got better data if they didn't tell the fixit crew that a test was coming. ------------------------------ From: eugene@AMES-NAS.ARPA (Eugene Miya) Date: 7 Feb 1986 0849-PST (Friday) To: RISKS FORUM (Peter G. Neumann, Coordinator) Subject: Re: RISKS-2.7: Earthquake monitoring systems ... I can tell you that earthquake instrumentation really need not survive a local earthquake. Local measurement is very unreliable because of environmental factors: soil type, underlying geologic structures, and so forth . Meters over such areas go off the scale as a matter of practice. It's the more distant meters which can separate the different waves which are important for triangulating location, magnitude, etc. --eugene miya, NASA Ames Research Center, eugene@ames-nas {decwrl,hao,ihnp4,hplabs}!ames!aurora!eugene ------------------------------ End of RISKS-FORUM Digest ************************ -------