Subject: RISKS DIGEST 11.16 REPLY-TO: risks@csl.sri.com RISKS-LIST: RISKS-FORUM Digest Monday 25 February 1991 Volume 11 : Issue 16 FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: RISKS in radiation treatment of cancer (Peter Kendell) Computer Tax Glitch in Los Angeles (Steve Milunovic) Computer problems with MD-11 jumbo jet (Fernando Pereira) Re: Warranties (Jim Horning) Accuracy in movies and newspapers (Tom Neff) Re: Biology of Unix; Message attributions (Michael Travers) Re: Worse-is-better for the 1990s (Unix) (Joseph M. Newcomer) Peace Shield and "software development problems" (Henry Spencer) Monopoly Security Policies for Thumb Prints (Bob Baldwin) Re: More on very reliable systems (Rod Simmons, Robert I. Eachus) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, and nonrepetitious. Diversity is welcome. CONTRIBUTIONS to RISKS@CSL.SRI.COM, with relevant, substantive "Subject:" line. Others ignored! REQUESTS to RISKS-Request@CSL.SRI.COM. FTP VOL i ISSUE j: ftp CRVAX.sri.comlogin anonymousAnyNonNullPW CD RISKS:GET RISKS-i.j (where i=1 to 11, j is always TWO digits. Vol i summaries in j=00; "dir risks-*.*" gives directory; "bye" logs out. ALL CONTRIBUTIONS CONSIDERED AS PERSONAL COMMENTS; USUAL DISCLAIMERS APPLY. Relevant contributions may appear in the RISKS section of regular issues of ACM SIGSOFT's SOFTWARE ENGINEERING NOTES, unless you state otherwise. ---------------------------------------------------------------------- Date: 23 Feb 91 10:07:42 GMT (Sat) From: Peter Kendell Subject: RISKS in radiation treatment of cancer >From the Guardian (London edition) Saturday February 23rd Patients die after radiation mix-up A Spanish health official said yesterday that he feared the worst for 24 cancer patients who received high doses of radiation when a Zaragoza hospital's linear accelerator went haywire for 10 days. Three patients have already died. A Zaragoza judge is investigating the causes of the disaster, which the director of the Insalud chain of state-run hospitals called "the worst accident in the world" of its type. "We fear the worst for some of the patients," an Insalud spokesman, Fernando Gomez, said. "We are seeking information to find someone who has experience in this kind of situation." General Electric officials were unable to comment on the machine's operation. Hospital officials said that the machine was now functioning normally. Peter (pete@tcom.stc.co.uk) ------------------------------ Date: 25 Feb 91 10:31:19 U From: "Steve Milunovic" Subject: Computer Tax Glitch in Los Angeles [abridged by PGN] A story by Denis Wolcott (with contributions from Walter Hamilton) in today's Los Angeles Daily News from the NY Times service described how thousands of LA County homeowners were billed up to $15,000 for three years' property taxes, because of a 1988 glitch in an $18M computer system (`Optimum'), which did not work and had to be rewritten from scratch. As a result, the county was unable to collect $10M in taxes, and has kept them from disbursing the tax money to schools districts and other agencies. Mercifully, the county will not charge back interest on the unbilled back taxes! ------------------------------ Date: Sun, 24 Feb 91 14:55:19 EST From: pereira@klee.research.att.com (Fernando Pereira) Subject: Computer problems with MD-11 jumbo jet According to the AP (2/20/91), American Airlines might refuse delivery of a second MD-11 jet from McDonnell Douglas because of computer and fuel problems with the first MD-11 it received. American's chairman said that the airline is ``very, very, very unhappy'' with the plane. American has suspended flight certification tests for the plane because of the computer problems. ------------------------------ Date: Thu, 21 Feb 91 12:31:18 PST From: horning@src.dec.com (Jim Horning) Subject: Warranties I, too, consider the standard shrinkwrap warranties scandalous, and definitely a RISK to the public. But it's not easy to see how to fix them. Even though the function of hardware is much better specified, computer hardware generally comes with what I call the Kodak Film Warranty: "liability limited to cost of unexposed film." The current round of discussion has relied too much on strained analogies with automobiles, and not enough on a rational discussion of the issues. Greg Johnson and Gene Spafford separately made two important points in RISKS 11.15. Taken together, they provide a basis for considering what a sensible warrantee should promise. 1) Software isn't a "thing" that can be purchased, it is a design. As Dijkstra has said, "The true subject matter of the programmer is the design of computations." If we pursue the automobile analogy, we need to consider, not defective automobiles, but defective automobile designs (e.g., Pinto gas tanks). 2) Software doesn't wear out, and hence doesn't need "maintenance" to cope with physical wear. Any defects were present at the outset. I think publishing provides a more natural and enlightening analogy: - We buy media containing software, just as we buy CD's containing music or books containing words. - There is a sharp distinction between the medium being defective (lost bits or missing pages) and the contents being defective. - Most publishers warrantee only the medium, not the contents. However, if a CD was labeled as Beethoven's Fifth Symphony and the contents turn out to be 2 Live Crew, you can probably get an exchange during the first week after purchase, even if you've broken the shrink wrap. - The medium is subject to physical wear and decay, not the contents. - Some publishers of certain kinds of books will, for a fee, provide periodic updates to the contents of their books. Most book buyers don't pay for such services, but they are invaluable for a few. Other publishers will supply errata on request. - There are few legal restrictions on what purchasers may do with the medium, many more about what they may do with the contents. Not much software is sold with a specification precise enough to allow a customer to prove it "doesn't perform as specified." Maybe someone who is outraged at the security hole that started this discussion would post the part of his vendor's specification stating that no such security holes exist? Failing that, maybe someone would post a specification that the vendors "ought to have" based their warrantees on? After we see these specifications, we can discuss: - What fraction of the total functionality of the operating system this component of the specification represents. We need some idea of the size of the total specification that a warrantee should be based on. - What fraction of the outraged customers would have noticed if JUST THIS PIECE of the specification had been omitted by their software vendor. And what fraction of those who noticed would then have refused to buy it. Until we have ANSI standards (or their equivalents) for complete operating systems and application packages, I'm afraid that the ordinary customer is at the mercy of the competence and goodwill of the software vendors. Jim H. ------------------------------ Date: 21 Feb 91 00:00:14 GMT From: tneff@bfmny0.bfm.com (Tom Neff) Subject: Accuracy in movies and newspapers (Re: Hollombe, RISKS-11.15) The beauty of TV and newspapers is that _everything_ in them is wrong! This is really true. No matter what the subject matter is, if you happen to be a specialist in that area, you'll grit your teeth when you read or watch. Neurosurgery -- ballet -- the law -- names of streets in your own hometown -- archaeology -- accounting -- you name it, they get it wrong. I'm not surprised: the media have to talk about everything under the sun but couldn't possibly afford to be experts in all of it. The fun part is that even when we're done rolling our eyes at, say, some egregious astronomy error, we sit back and take at face value something about China or Churchill or Chernobyl or child development! We shouldn't. Experts in those areas are busy gritting their teeth even now -- while they swallowed the astronomy stuff without complaint. :-) [Another instance that strikes home even more is being directly MISquoted after making a carefully worded direct statement and insisting that it be used verbatim if at all... Perhaps there is nothing special about computers and related technologies that causes many media folks to be so far off the mark. But there are lots of technological nonsophisticates writing on technology (and only a few really thoughtful and careful ones). Perhaps the worst problem is the tendency toward 10-second sound bites and 25-words-or-less oversimplifications. PGN] ------------------------------ Date: Thu, 21 Feb 91 15:39:14 EST Subject: Re: Biology of Unix; Message attributions From: mt@media-lab.media.mit.edu Sorry, but I'm not the author of that message, it was forwarded from some company's internal mail system and the best guess at the author ID is MADRE::MWM. My own opinion of Unix is that it's more like crabgrass, or rabbits in Australia -- a rapidly-spreading and obnoxious weed that invades computational ecologies and displaces the native species. Also, the Mike that signed the first few paragraphs is not me as you inferred; it must be MWM. I only wrote the first sentence. [finger informs me that "mt" is Michael Travers. Apparently he was unaware that HIS OWN FROM: field did not give his own name! In response to a poke from me, he has now upgraded. PGN] ------------------------------ Date: Thu, 21 Feb 1991 15:32:30 -0500 (EST) From: "Joseph M. Newcomer" Subject: worse-is-better for the 1990s Here we are in 1991. The two primary operating systems (at least in volume) are representative of O/S technology of the late 1960s to early 1970s (Unix and MS-DOS), and the three important languages are C (vintage mid-1960s, i.e., it is BCPL in disguise), LISP (vintage late 1950s) and Ada (vintage early 1970s). With the exception of LISP, there have been many better operating systems and languages lost along the way, and there seems to be no interest in updating our technology. I have my list of better languages and O/Ss (and it is probably different than yours). But on the whole, the barely-adequate-but-portable has replaced the not-too-bad or even pretty-good but-not-portable. There is a lesson here. There is a risk of accepting the barely adequate; you may have to live with it for a long time. ------------------------------ Date: Mon, 25 Feb 91 14:45:19 EST From: henry@zoo.toronto.edu Subject: Peace Shield and "software development problems" A friend, who has asked not to be identified, says it looks to him like the Boeing problem with Peace Shield was not software-development trouble but cost estimation. (He has worked for Boeing but is not currently a Boeing employee.) His feeling is that Boeing bid low on a fixed-price contract, discovered an impending cost overrun, found that the Saudis weren't interested in pumping in more cash, decided that the best way to cut its losses was to encounter fatal software-development problems, and more or less stopped work on the project until the schedule slippage became too severe to ignore and the contract was cancelled. This explanation is certainly plausible. Whether accurate or not, it points out a significant issue: problems in developing large software systems are sufficiently common that it's convenient to use them as an excuse when something goes wrong elsewhere. It's probably wise to be very cautious about blaming the software people for recent military systems, in particular. DoD has recently been using fixed-price contracts a lot, and a good many military contractors have discovered -- the hard way -- that they've forgotten how to do realistic cost estimates. Since DoD basically has no memory, it's better to default on the contract and accept some transient bad feelings than to fulfill it and lose money. Software makes a wonderful excuse, since it's considered a natural law that a certain fraction of big software projects inexplicably fail with nobody to blame. Henry Spencer at U of Toronto Zoology utzoo!henry ------------------------------ Date: 25 Feb 91 11:53:00 -0800 From: Bob Baldwin (baldwin_bob#tsii@tandem.com) Subject: Monopoly Security Policies for Thumb Prints The California department of motor vehicles requires a right thumb print to get a driver's license or state ID card, so the DMV now has a large online database of thumb prints. The primary purpose of this database is to prevent people from taking on new identities without DMV knowing about it (and thus getting clean driving records). What if someone is supposed to have a new identity? What about witness relocation programs? What about undercover police officers? The DMV has to treat all its thumb print data as being as sensitive as the most sensitive thumb print it contains. One alternative is for each government agency submit a list of the thumb prints that need special restrictions. Needless to say, the DMV wouldn't want to pay for the safeguards that would be required on such a list. The underlying problem is the desire to enforce security (access) policies that imply monoploy control of data. For example, the FBI wants to know about all queries matching the finger prints of its employees. The National Crime Information Center can support this policy, but such a policy is hard to enforce when different states are involved. The FBI doesn't what to tell all the states about all its employees. What's the solution? We could eliminate monopoly security policies and the programs that depend on them. We could have the federal government maintain its monopoly, and provide services to the states. We could trust the states to make sure that only "good guys" access the data. Perhaps the best solution is to ignore the problem and go to lunch. ------------------------------ Date: Sun, 24 Feb 91 11:56:15 -0500 From: rod simmons Subject: Re: More on very reliable systems (Leichter, RISKS-11.12) >In fact, I've always found it interesting how much more we demand from digital >systems than we demand from mechanical ones. ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Perhaps we "do" because we "can," or at least we think that we "can," based upon an inspection of failure probabilities for various types of components and devices (such as those given in WASH 1400, and other similar compilations of failure data). --Rod Simmons ------------------------------ Date: Mon, 25 Feb 91 17:11:12 EST From: eachus@d74sun.mitre.org (Robert I. Eachus) X-Alternate-Route: user%node@mbunix.mitre.org Subject: Re: More on very reliable systems (Siegman, RISKS-11.14) Anthony E. Siegman (siegman@sierra.Stanford.EDU) said: > Anyone concerned with the subject of multiple correlated > failures in systems with very reliable individual components should > look back at the incident some years ago when a United Airlines jet > lost all three engines simultaneous in flight over the Caribbean > south of Miami.... > 2. The problem was really a false alarm, i.e., the oil > pressures were OK, just the sensor indications were wrong. No, the engines were really overheating because all the oil leaked out past the (missing) seals. > Oh, they did get one (or two?) of the engines restated, just in the > nick of time, however, and limped into Miami. When the middle engine started to overheat, the pilot shut it down, declared an emergency, and immediately headed back toward Miami, grabbing for altitude. When the other two engines showed overheat, I think he cut the RPMs back so that engine failure would not be catastrophic, but in any case ran them to failure. When these two engines failed he was on a straight in glide to Miami International, with calculations showing that at best glide, the plane would hit the water a mile short of the runway. Passengers were told to prepare for the possibility of a water landing. At an altitiude of 200 feet as planned (and apparently as explained to the passengers) the crew restarted the middle engine (which they had shut down remember) and got the hoped for two minutes of life out of it, long enough to land and get off the main runway. All three engines were damaged beyond repair. Risks (or lessons learned) from this and the Gimli Glider incident? First, airline pilots should be required to have glider experience. In both these cases, the pilots knew what to do, however there have been several cases where the aircrew behavior during a multiple engine shutdown has verged on panic, most recently a Chinese 747. (There have been lots of such incidents, usually involving flying through clouds volcanic origin.) Also, and much more important, but often overlooked, is the value of experience in knowing when some condition is "new" unknown and the trust which should be given to the pilots by their employers (and the FAA) in such cases. What would have happened if the pilot in this case took his time about turning back to see if he had a "real" ememrgency? In this case he acted immediately to turn around--he could always turn around again if the problem wasn't serious, and expect the airline to back his judgement. With some airlines, the pressure is in the opposite direction (schedule ahead of safety) and that was a contributing factor in the Gimli glider incident. ------------------------------ End of RISKS-FORUM Digest 11.16 ************************