Subject: RISKS DIGEST 10.57 REPLY-TO: risks@csl.sri.com RISKS-LIST: RISKS-FORUM Digest Sunday 4 November 1990 Volume 10 : Issue 57 FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: $6.3 million electric bill (Mark W. Schumann) Drug RISKS to software ?? (Simon Rosenthal) More of what really goes wrong (John Rushby) Automotive electronic engine control failure modes (Dave Davis) Re: Laxness, not modernization, at fault in train wreck (Chuck Weinstock) Train Wreck and Weight Estimates (anonymous) Re: Risks of Modernization (Gerald Stafleu) Re: Access to gov't computer files (Brinton Cooper) Call for papers -- ACM SIGSOFT '91 (Nancy Leveson) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, and nonrepetitious. Diversity is welcome. CONTRIBUTIONS to RISKS@CSL.SRI.COM, with relevant, substantive "Subject:" line. Others ignored. REQUESTS to RISKS-Request@CSL.SRI.COM. FTP VOL i ISSUE j: ftp CRVAX.sri.comlogin anonymousAnyNonNullPW CD RISKS:GET RISKS-i.j; j is TWO digits. Vol summaries in risks-i.00 (j=0); "dir risks-*.*" gives directory; bye logs out. ALL CONTRIBUTIONS CONSIDERED AS PERSONAL COMMENTS; USUAL DISCLAIMERS APPLY. Relevant contributions may appear in the RISKS section of regular issues of ACM SIGSOFT's SOFTWARE ENGINEERING NOTES, unless you state otherwise. ---------------------------------------------------------------------- Date: Fri, 2 Nov 90 19:09:53 EST From: catfood@ncoast.org (Mark W. Schumann) Subject: $6.3 million electric bill Whoops! CEI bills customer $6.3 million by Sabrina Eaton, Staff Writer The Plain Dealer [Cleveland] Friday, 2 November 1990 Even when here cupboard was bare during the Depression, 76-year-old Faye Starman always paid her electricity bills. That was before Tuesday, when she opened here $6.3 million Cleveland Electric Illuminating Co. bill. "It's a good thing I'm a calm person because I could have had a heart attack or stroke," said Starman of Newbury Township. The bill, payable by Nov. 9, also gave her the option of making a $300,021 budget payment--far more than her home on Ravenna Rd. is worth. CEI spokesman Mike Lumpe said the company's records were corrected immediately after her complaint and blamed the slip on mistakes made during a switch to a new computer billing system. "The employee had tried to enter $63 in our computer, made a mistake, and the computer filled in the extra zeroes automatically," Lumpe said. "People are learning the new system, and there are bound to be one or two mistakes, especially when you send out 750,000 bills every month." Lumpe said the company also has heard from an Ashtabula County family that received a $2 milion bill, and said he hoped "we don't have any more of those floating around out there." Only a few of CEI's industrial customers have monthly bills in the $1 million range, but Lumpe said he did not believe there were any customers with bills as high as $6 million. I am curious about two obvious problems here. First of all, shouldn't there have been a "sanity check" for a reasonable range of billing amounts on the data entry application? But secondly, what modern billing system would require manual entry of the bill amounts in the first place? Here we aren't dealing with a risk of computerization; this is over-reliance on manual procedures without adequate controls. Mark W. Schumann 3111 Mapledale Avenue, Cleveland 44109 USA Domain: catfood@ncoast.org ...!mailrus!usenet.ins.cwru.edu!ncoast!catfood ------------------------------ Date: Tue, 30 Oct 90 14:51:51 EST From: simon@westford.ccur.com Subject: Drug RISKS to software ?? Two occurrences today left me wondering of the RISKs to correct software of substance abuse by its authors. Today, I attended a presentation by my employer (Concurrent Computer Corporation) of its "drug free workplace" policy. In common with other companies which do business with the government, Concurrent is now mandated by Federal law to maintain a drug-free workplace - establishing and publicizing a policy, offering help or disciplining offenders, and reporting people convicted of drug offences to the government. Also today, the Boston Globe(10/30/90) reported a Supreme Court decision upholding an award to a programmer who was fired for refusing to provide a urine sample. " "The court declined to review a lower court ruling that the programmer, Barbara Luck, was not in a safety-related job for the railroad company (Southern Pacific) and could not be required to take the drug test" Southern Pacific's argument was that federal railway labor laws (requiring mandatory drug testing) should apply to all employees. The description of the programmer's job as "non safety related" led me to think of all the reports of failure of mission critical software which are regularly described in RISKS, and to speculate on whether the checks and balances usually present in the software development process (debugging, code reviews, Software Quality Assurance procedures) could ever be insufficient to catch software errors caused by drug-altered states of consciousness. Like many of us, I'm certainly aware of the mood-altering and productivity-diminishing effects of such drugs as alcohol (although in my experience equally negative effects can be caused by such things as inadequate sleep, or by pulling an all nighter). I tend to doubt whether software malfunctions could ever be attributed solely to drug abuse on the part of its authors. But ... I'd be interested to know if there have ever been any reports to the contrary. Simon Rosenthal, Concurrent Computer Corporation, Westford, MA 01886 ------------------------------ Date: Mon 29 Oct 90 22:19:22-PST From: John Rushby Subject: More of what really goes wrong >From Aviation Week, October 22, 1990 ``Engine Shutdown, Computer Glitch Delay YF-22A Test'' page 115 ``Officials had been hoping to raise the landing gear---for the first time---on the second flight Oct. 11, but two computers disagreed with one another and prevented gear retraction. The gear is extended by electrical commands without computer intervention.'' ``Rockwell/MBB X-31 Makes Second Flight, Reaching 20,000-Ft. Altitude, Mach 0.6'' page 117 ``The electronic flight controls had several internal disagreements on the second flight that led to the use of reversionary modes, but the disagreements cleared when the computers were reset. Testing was continued and the aircraft landed in normal mode.'' John ------------------------------ Date: Tue, 30 Oct 90 08:18:41 EST From: dave davis Subject: Automotive electronic engine control failure modes i was a product design engineer with Ford in the late 70s through early 80s and I can relate a story of electronic engine control failure that was subtle and embarrassing to us. The first models to be equipped with our EEC-1 system were those with the big V-8s, including fleet sales, such as taxis, rental agencies, and police. We had a system of reporting difficulties that dealer mechanics had with diagnosing and repairing our cars, so that engineering and management could be aware of the need to correct design or manufacture. One of our regional dealer reps. reported that a police car equipped with EEC-1, had experienced complete loss of engine operation, that is, ignition, each time the officer keyed the mike button on his radio! After a lengthy investigation, our people at the dealer found that a single ground strap from the hood to the firewall had been bent during manufacture or shipping at wasn't doing its job. This had apparently allowed enough RF to leak into the engine compartment, to be picked up by the EEC-1 system's wiring, and fed into the control module (a Motorola computer chip) where it played hell with its operation. After much discussion at the highest levels, we were forced to recall every police vehicle sold to date with EEC-1, because of safety--real safety, not theoretical. (The engineer responsible for the sensors recommended early on that the cars had to be recalled. He complained to me 'They stopped inviting me to the meetings.') The last time I visited the office where the EEC engineers were located, they had a prototype of a new control unit with through the wall capacitors--about 36 of them. This was required to filter the RF, for police buyers and anyone else who carried a powerful radio transmitter. All of this in a business where one is a hero if you can shave a nickel out of producing several thousand cars. Dave Davis, MITRE Corporation, McLean, VA me := disclaimer.all ------------------------------ Date: Mon, 29 Oct 90 16:24:48 EST From: Chuck Weinstock Subject: Re: Laxness, not modernization, at fault in train wreck In Risks Digest 10.56 various people commented that blaming modernization as a cause of the train wreck was not correct. For instance, Roy Smith said: "The implication here, that old mechanical scales are safe and new (presumably) computerized scales are dangerous, seems far out of line with the facts presented. The crash occurred because the train was overloaded and because they only had half the braking capacity they thought they had, both bits of misinformation due to plain old poor operating practices, not fancy modern scales." Actually, I have no complaint with using more modern, computerized scales. My comment was that if the railroads had not gone to the more expensive computerized technology, they might still have had scales at more locations including at Mojave. Certainly an old technology mechanical scale at the top of the hill would have helped more than a modern technology scale at the bottom of the hill. This contributes to the cavalier attitude towards of risks of their actions exhibited by many of the people involved. Certainly there were other contributing factors in the crash such as those pointed out in my original post and expanded upon (and added to) by Martin Minow and others. Chuck Weinstock ------------------------------ Date: Mon, 29 Oct 1990 xST From: [anonymous] Subject: Train Wreck and Weight Estimates In a recent message, it was pointed out that it was only through the intervention of a human that the correct number of locomotives was chosen to pull the load in question, even though the load weight had been badly underestimated. However, in this case, I wonder if that fellow basing the load on his own estimates, rather than the provided information, didn't actually contribute to the accident (though with the best of intentions). If he had believed the underestimates presumably provided to him, he would probably have assigned fewer locomotives to the train, and the fact that the load was "too heavy" would have likely been noticed by the crew almost immediately, or at least well before they reached the grade in question. By providing the "proper" number of locomotives for the actual load, the train was able to proceed easily to the grade where the lack of sufficient braking became critical. Of course, his using "correct" estimates, rather than the "low" estimates, only became a problem due to the complex of other interacting factors involved with the incident. ------------------------------ Date: Tue, 30 Oct 90 08:55:04 EDT From: Gerard Stafleu Subject: Re: Risks of Modernization (Trona Train Troubles) davidsen@crdos1.crd.ge.com writes: > Do these trains run with no normal air brakes on every car? That seems to be a more relevant question here than electronic scales. I thought it was Standard Operating Procedure that train cars should be able to out-brake the locomotive(s). If they cannot do so, and the locomotive starts to out-brake the cars, the cars will all pile neatly on top of the locomotive. This will indeed stop the train, but not in a manner according to specs. In other words, the total weight being pulled by the locomotives does not (or should not) have any influence on the braking capacity needed for each locomotive. Each locomotive should be able to stop itself, as should each car. This concept is not unfamiliar to computer people: while global error detection and correction is desirable, that does not mean that each module should not also do its own checking and correcting. Gerard Stafleu (519) 661-2151 Ext. 6043 BITNET: gerard@uwovax ------------------------------ Date: Mon, 29 Oct 90 23:28:50 EST From: Brinton Cooper Subject: Re: Access to gov't computer files (Sullivan, RISKS-10.56) John Sullivan reported on a Freedom of Information request for statistical info on real property in NYC. The city resisted by offering it in a million sheets of hardcopy form for about $10,000 . A court ruled that NYC had to provide it on magtape at a cost under $100. John calls attention to a " concern about possible problems involved in making too much computer data available..." For many years, NYC has been making property transaction records available, quarterly, on magtape to anyone who will pay the nominal copying fee. They are used on PC-based, Novell-networked type systems (and others, no doubt) by people who do many title searches for themselves or as a service-for-fee for others. The manual searching of titles in file cabinets at City Hall would be prohibitively expensive, considering the size of the database. A manual title search involves beginning from the most recent deed and tracing the sequence of sales as far back as possible, going from one "liber and folio" to another, to another, etc. It's difficult to imagine how, in a city the size of New York, such a necessary activity could be carried out without the widespread use of automated searches. It's equally difficult to imagine that the manual, labor-intensive method would be less prone to make undetected errors than would the computerized system. _Brint ------------------------------ Date: Mon, 29 Oct 90 17:45:06 -0800 From: Nancy Leveson Subject: call for papers -- ACM SIGSOFT '91 CALL FOR PAPERS ACM SIGSOFT '91 Software for Critical Systems New Orleans, Louisiana December 4-6, 1991 Computer systems are beginning to affect nearly every aspect of our lives. Examples include programs that control aircraft, shut down nuclear power reactors in emergencies, monitor hospital patients, and execute banking transactions. Although such programs offer considerable benefits, they also pose serious risks in that we are increasingly vulnerable to errors and deficiencies in the software. The SIGSOFT '91 conference seeks papers on all aspects of quality in critical systems. A critical system is a system that must exhibit, with very high assurance, some specific qualities such as safety, reliability, confidentiality, integrity, availability, trustworthiness, and correctness. The conference will focus on such topics as architectures, design methodologies, languages, analysis techniques, and processes that can increase the likelihood that a system exhibits its required qualities. Papers will be judged on relevance, significance, originality, correctness, and clarity. Papers will be read and evaluated by the program committee and must not be under consideration (or published) elsewhere in the same or similar form. Papers are limited to 6,000 words, with full-page figures counting as 300 words. A paper that significantly exceeds this limit is likely to be rejected. Authors should submit 6 copies of the full paper to: Peter G. Neumann, Computer Science Laboratory, SRI International, Room EL-243, 333 Ravenswood Ave., Menlo Park, CA 94025 Persons submitting papers from countries in which access to copying machines is difficult or impossible may submit a single copy. Submissions should be received by May 3, 1991 and should include a return mailing address. Authors will be notified of acceptance or rejection by July 12, 1991. Full versions of accepted papers must be received in camera-ready form by August 30, 1991. Authors of accepted papers will be expected to sign a copyright release form. Proceedings will be distributed at the conference and will subsequently be available from ACM. CONFERENCE CHAIR PROGRAM Co-CHAIRS Mark Moriconi Nancy Leveson Peter Neumann SRI International Univ. of California, Irvine SRI International moriconi@csl.sri.com leveson@ics.uci.edu neumann@csl.sri.com PROGRAM COMMITTEE David Barstow Schlumberger Dines Bjorner Technical University of Denmark Marie-Claude Gaudel Universite de Paris - Sud Jim Horning DEC Systems Research Center Bill Howden University of California, San Diego Hermann Kopetz Technical University of Vienna Carl Landwehr Naval Research Laboratory Bev Littlewood City University, London Leon Osterweil University of California, Irvine David Parnas Queen's University Fred Schneider Cornell University Vicky Stavridou University of London Martyn Thomas Praxis, Inc. Walter Tichy University of Karlsruhe Elaine Weyuker NYU Courant Institute ------------------------------ End of RISKS-FORUM Digest 10.57 ************************