RISKS-LIST: RISKS-FORUM Digest, Tuesday, 12 August 1986 Volume 3 : Issue 36 FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: Another Medical Risk? (Lee Breisacher) RISKy Business in Surgery (Mark Jackson) Reliance on word-processors discussed in the Israeli Supreme (Ady Wiernik) Expert Systems - The New Cop on the Beat (Laws via Fred Ostapik) Chernobyl (Art Evans, Dick Karpinski) Air Traffic Control computer failure (Dan Melson) Possible failures of BMD software (Herb Lin) A note about stories "from memory" (Henry Mensch) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, nonrepetitious. Diversity is welcome. (Contributions to RISKS@CSL.SRI.COM, Requests to RISKS-Request@CSL.SRI.COM) (Back issues Vol i Issue j available in CSL.SRI.COM:RISKS-i.j. Summary Contents in MAXj for each i; Vol 1: RISKS-1.46; Vol 2: RISKS-2.57.) ---------------------------------------------------------------------- Date: 12 Aug 86 09:25:49 PDT (Tuesday) Subject: Another Medical Risk? From: Breisacher.OsbuSouth@Xerox.COM To: RISKS@CSL.SRI.COM From the August PSA Airline magazine, extracted from an article about inventors: [There's a photo of Dr. Kwoh in surgical garb in an operating room leaning over a dummy patient with some elaborate equipment surrounding its head. The caption reads:] Robotic surgery is a reality because of the obsessive work of Yik San Kwoh, medical research and development director of Long Beach Memorial Hospital. His computer controlled "surgeon," capable of conducting brain surgery within an accuracy of 1/2000 of an inch, was the result of three years of incessant programming. [From the text of the article:] Yik San Kwoh, medical research and development director of Long Beach Memorial Hospital, explains, "I've got two Apple computers at home and three IBMs. I spend so much time on those damn things that I get sick of it. Only then can I stop." It took three years of programming and reprogramming for Kwoh to turn and industrial robot into a surgical instrument capable of conducting brain surgery. [As usual, we must weigh the risks of using such equipment against the risks of NOT using it. On the other hand, the description makes it sound like he programmed this thing the way I wrote my first couple programs (FORTRAN in the early 70's) -- dive in and start writing code then keep debugging til it sorta works.] Lee --------------------------------- Date: 12 Aug 86 07:56:26 EDT (Tuesday) From: MJackson.Wbst@Xerox.COM Subject: RISKy Business in Surgery To: RISKS@CSL.SRI.COM From /Programmers at Work (1st Series): Interviews/, by Susan Lammers (Microsoft Press, 1986): "My most amazing experience, though, was a phone call I got right after I started Iris, from a surgeon who was using Symphony for real-time data analysis during open heart surgery. It is sobering to think that someone was lying on an operating table potentially relying upon my program running properly. It reminds one of the real responsibility to the end users." -- Ray Ozzie project leader for Symphony ------------------------------ Date: Tue, 12 Aug 86 21:19:31 -0300 From: Ady Wiernik To: risks@sri-csl.ARPA Subject: Reliance on word-processors discussed in the Israeli Supreme Court Rules of Court in Israel fix a time limit for bringing an appeal to the Supreme Court against a decision of an inferior Court. A lawyer applied to Supreme Court for an extension of the period to appeal. He has missed the statutory period by two days. His excuse was that the word-processor in his office (that has been recently installed) malfunctioned. The text of the appeal that was typed into the computer has been erased because of that computer malfunction. He called the maintenance personnel. They promised that the malfunction would be shortly repaired, but actually, it lasted longer, causing him not to be able to bring the appeal at the same day. The appellant claimed that the trouble with the computer was an "act of god", Force Majeure, which is considered a special ground that entitles him the desired extension. The court has rejected this argument. In his judgement, Registra Tzur of the Supreme Court said: "Indeed, the computer is very useful, but one must prepare for possible malfunctions in its operation. When there is no computer, the good old typewriter should replace it." This decision is the first recorded judicial reference to the use of word-processing devices in lawyer offices, and displays the dangerous results of reliance on high-tech. Ady Wiernik ------------------------------ Date: Mon 4 Aug 86 22:38:23-PDT From: Laws@SRI-STRIPE.ARPA [courtesy of Fred Ostapik] Subject: Expert Systems - The New Cop on the Beat The FBI has developed Big Floyd, an expert system to assist in criminal investigations. Similar programs are being developed to catch drug smugglers and target potential terrorists. The EPA wants to identify polluters; the Treasury Department is looking for money-laundering banks; the Energy Department would like to find contractors who cut corners; the Customs service is after drug smugglers; the IRS is developing a system to spot tax cheaters; the Secret Service is working on a classified system to point out potential presidential assassins; and the FBI's National Center for the Analysis of Violent Crimes is developing expert systems to identify potential serial killers, arsonists, and rapists. Systems to target counterfeiters and bombers are also being built. -- Michael Schrage, The Washington Post National Weekly Edition, Vol. 3, No. 40, August 4, 1986, p. 6. ------------------------------ Date: Tue 12 Aug 86 11:34:21-EDT From: "Art Evans" Subject: Chernobyl To: Risks@CSL.SRI.COM In RISKS-3.35, Robert Stroud comments on "Official Report on Chernobyl disaster". Although the discussion of what actually triggered that disaster is interesting, I choose to focus instead on how the Russian explanation was interpreted by others (not by Mr Stroud). Quoting from the post: But many believe the explanation [offered by the Russians] is inadequate and that it is being promoted mainly to protect the country's nuclear construction programme. No justification is given for this belief. A Peter Potter is quoted as saying By maintaining that human error and turbine problems were really to blame, the Russians could say that their reactors have no serious design flaws. They could then avoid calls for closures of other reactors or for the implementation of drastic redesign work. This claim may in fact be true, but we are given no evidence. Note what is happening: The Russians offer a technical explanation for the disaster. A western nuclear expert says the explanation is inaccurate and was offered for political reasons. But, no reason other than political is given for this skepticism. The Russians may well be lying, and if there is evidence I would like to see it. Lacking such evidence, though, the public would be better served by less misleading pronouncements by "experts". Art Evans ------------------------------ Date: Tue, 12 Aug 86 11:13:17 PDT From: dick@cca.ucsf.edu (Dick Karpinski) To: risks@csl.sri.com Subject: Chernobyl The only unadvertised design deficiency that I know of in the Chernobyl reactor is that it has a positive coeficient of reactivity with respect to temperature. That is, when the temperature goes up, so does the rate of nuclear fission. Such a design would be ruled out here, claims my source, a former reactor containment vessel engineer. Surely, such a design would make the sort of accident which occurred more likely. Dick Dick Karpinski Manager of Unix Services, UCSF Computer Center UUCP: ...!ucbvax!ucsfcgl!cca.ucsf!dick (415) 666-4529 (12-7) BITNET: dick@ucsfcca Compuserve: 70215,1277 Telemail: RKarpinski USPS: U-76 UCSF, San Francisco, CA 94143 ------------------------------ Date: Mon, 11 Aug 86 23:47:21 PDT From: crash!pnet01!dm@nosc.ARPA (Dan Melson) To: crash!noscvax!RISKS@CSL.SRI.COM Subject: Air Traffic Control failure Computer failures at Air Route Centers are not as uncommon as we'd like, but they're not as nasty as they could be. Despite the fact that the computers currently used are more than fifteen years old, they seem to handle the load well enough for the present. When the primary computers (IBM 9020's) go down, however, the DARC backup system does not furnish the controllers with nearly as much data, and it is far more difficult to get automated tasks done. There is currently a new computer system in the works, and when it is operational, delays due to computer failure should dramatically decrease. The estimate for this is 'around 1990'. At any rate, even the bachup systems are far more pleasant than doing all of the work manually. DM ------------------------------ Date: Tue, 12 Aug 1986 00:38 EDT From: LIN@XX.LCS.MIT.EDU To: risks@CSL.SRI.COM Subject: Possible failures of BMD software I'm working on a paper on potential software-induced difficulties and problems that might accompany the deployment of a BMD system. I'd like to enlist the collective imagination of the list on examples apropos to this paper. Please constrain your imagination by the limits of the possible (e.g., it is impossible for an X-ray laser to shoot x-rays at ground targets, but it is not impossible that the firing of an X-ray laser creates an electromagnetic pulse that has unanticipated effects). Please specify the scenario in as much detail as you can. I am not specifying a system architecture, so please tell me the one(s) you have in mind in your scenario(s); that is necessary because softare -- by itself -- is harmless no matter how buggy it is. Also remember that BMD has significant capability against satellites. Thanks. Acknowledgements will be provided if you so desire. Herb Lin ------------------------------ To: risks@csl.sri.com Subject: A note about stories "from memory" Date: Mon, 11 Aug 86 23:44:12 -0500 From: Henry Mensch I hate to sound like a nit-picker but I've noticed a rash of stories which begin with words like "If I remember correctly ..." or "It's pretty late, so expect errors." Is this sort of thing a product of having such powerful communications tools at our fingertips? Once these things happen we seem to spend a lot of time saying "Well, *I* thought it went this way. . . " In discussing risks to the public, we risk wasting our time doing these tasks, which could be avoided with a bit of research. Striving for better communications, - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Henry Mensch | Technical Writer | MIT/Project Athena henry@athena.mit.edu ..!mit-eddie!mit-athena!henry [On the one hand, it is nice to be precise. On the other hand, if the report is novel and interesting, perhaps RISKS provides a medium for getting feedback from an expert on a matter that would otherwise go unreported. But, I certainly appreciate it when contributors take a little time to track down the reference -- and especially when they cite that reference. PGN] ------------------------------ End of RISKS-FORUM Digest ************************