RISKS-LIST: RISKS-FORUM Digest Saturday 11 November 1989 Volume 9 : Issue 41 FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: Stuffing the electronic ballot box (again) (PGN) BART and the Bartered-Computer Commuters Coral reef ruined by poor user interface design? (Jim Helman) Re: Computer errors and computer risks (Jerome H Saltzer) Computer used to find scoflaws in Boston (David desJardins) Reference on the early history of Ada -- killing reliably (Eugene Miya) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, and nonrepetitious. Diversity is welcome. CONTRIBUTIONS to RISKS@CSL.SRI.COM, with relevant, substantive "Subject:" line (otherwise they may be ignored). REQUESTS to RISKS-Request@CSL.SRI.COM. TO FTP VOL i ISSUE j: ftp CRVAX.sri.comlogin anonymousAnyNonNullPW cd sys$user2:[risks]get risks-i.j . Vol summaries now in risks-i.0 (j=0) ---------------------------------------------------------------------- Date: Sat, 11 Nov 1989 13:30:42 PST From: "Peter G. Neumann" Subject: Stuffing the electronic ballot box (again) Charles Schwab & Co spent more than $100,000 to conduct a nationwide poll on program(med) trading, with two widely advertised free 800 phone numbers set up to record the pro and con votes. Apparently someone at Wells Fargo (which has a high-profile program trading subsidiary) set up an autodialer to vote YES continuously, which was detected by their programmed monitoring. (` "First we have program trading. Now we have program dialing," quipped Senior Vice President Hugo Quackenbush. ') In 12 hours, there were 12,191 votes, 65.3% for, 34.7% against. [Source: San Francisco Chronicle, 11 Nov 1989, pp. B1-B2.] [By the way, I observe that one call every twenty seconds for 12 hours would account for the entire difference between the pros and cons. One call every six seconds would have accounted for ALL of the pro calls. We might suspect that an autodialer could easily have skewed the results -- although perhaps others on both sides were also using the same strategy.] ------------------------------ Date: Sat, 11 Nov 1989 13:46:02 PST From: "Peter G. Neumann" Subject: BART and the Bartered-Computer Commuters The Bay Area Rapid Transit has for years been having troubles with its computer systems -- both old and new. The old one is supposed to handle 64 trains at once, but is barely able to handle 39. The new system -- almost ten years late when it went on-line three weeks ago -- was supposed to be able to handle 108 trains, but becomes seriously overloaded at 10. Consequently, all of the long-planned extensions are on hold. (The system supplier is Logica. The old system is still being used, at least as a backup when the new one is being tested.) "An outside consultant that designed three control systems for the Paris Metro and was brought in by [BART General Manager Frank J.] Wilson has already determined that even if the new system were able to perform as promised, it is obsolete." [A summary of "BART's New Computer Called `Complete Failure'", By Harre W. Demoro, San Francisco Chronicle, 11 Nov 1989.] An earlier article indicated that the BART system has been sorely pressed in trying to accommodate the great increase in traffic due to the loss of the Bay Bridge in the earthquake, and many train cars are out of service. ------------------------------ Date: Fri, 10 Nov 89 19:09:37 -0800 From: helman@isl.Stanford.EDU (Jim Helman) Subject: Coral reef ruined by poor user interface design? The captain of a ship which ran aground on an environmentally sensitive live coral reef off the US coast attributed the accident to a confused officer and a bad user interface. Apparently, an officer incorrectly changed course because the steering mechanism on the ship operated in the opposite fashion from most such controls. Whether or not the control was computerized, this demonstrates quite dramatically the potential dangers of inconsistent user interfaces. I suppose the company which built the control could be held liable for the poor design. There may be parallels with GUI issues yet to come. What if a company has to use an inconsistent interface because of patent restrictions? Could the company still liable for the errors of a confused user? Excerpts from the UPI article follow: ``I think he made an error. I think he was confused,'' Capt. Zdravko Beran told Coast Guard investigators on the first day of testimony before a board of inquiry. ``He told me that (navigational) light was dead ahead and then he wanted to turn a few degrees to the port (left).'' Instead, the officer, Zvonko Baric, turned the ship to the right, or starboard, causing it to run aground on sensitive coral in the Fort Jefferson National Monument, Beran said. The steering mechanism must be turned in the opposite direction of the intended course -- the reverse of how most such devices operate, Beran said. Asked how future such accidents could be avoided, Beran said the federal officials could require ships to go around rather than through the national monument. Jim Helman, Department of Applied Physics, Stanford Univ., Stanford, CA 94309 ------------------------------ Date: 10 Nov 1989 1613-PST (Friday) From: saltzer@src.dec.com (Jerome H Saltzer) Subject: Re: Computer errors and computer risks (RISKS-9.40) In RISKS DIGEST 9.40, Randy Davis says, > Numerous stories have been reported on this list under the title > "computer error" and "computer risk," that seem to me to have nothing > essential to do with computers, and a great deal to do with very > different issues. > . . . I suggest the simple test above: Ask, can the identical > problem can arise in the absence of computers? I claim that it is not that simple. In a traditional library, it was possible to invade your privacy by making a list of all the books you have every checked out. All an investigator had to do was open every book in the library and look to see if you had signed the card inside. The information was publicly available, but actually it was benignly protected by an enormous collection cost, so noone every worried about it. These days, if the library has a computerized circulation system, and it is not designed with proper regard for privacy, it may be possible to get this information in a few seconds with a single request. Most computerized circulation system protect this information and many installations systematically delete it to avoid any possibility of such investigations. Yet, when the first circulation systems were designed, some people said that the circulation information has "always been public" as an argument against protecting the information in the computerized system. The Registrar of Motor Vehicles in Massachusetts has long used the "it has always been public" argument on car registration information as an excuse for selling tapes to mass mailers. But as I recall, I didn't receive that class of junk mail back in the days when you had to personally visit the Registry and copy the information out of their ledgers. The point is that the speed and efficiency with which data can be processed by computer can convert a neglible risk into one worth discussing. Had the RISKS forum been around then, I think the library debate would have been quite an appropriate topic. Ditto the Mass. Registry. Finally, the availability of the computer to discover connections may tempt people to create data bases that they wouldn't otherwise consider feasible. When I saw the report in RISKS on the proposed child-abuse data base I assumed that this was an example of a data base that probably wouldn't have been proposed had computers not been available to implement it--especially the part about identifying people who use multiple doctors. Thus I am prepared to tolerate a certain amount of fuzziness in identifying the edge of the computer-RISKS arena by our beleaguered moderator. Jerry Saltzer ------------------------------ Date: Fri, 10 Nov 89 21:01:38 EST From: desj%idacrd@Princeton.EDU (David desJardins) Subject: Computer used to find scoflaws in Boston >>When five out of six hits are human errors, imagine the complaints! > > It goes to show the importance of considering the total effect of a system > change, not just the project at hand. It was a serious design error [...] I am very surprised at how much complaining there has been about this error rate. I think finding one stolen car in every six stopped is a phenomenally *high* success rate. It doesn't seem a major problem to me to inconvenience five people for a relatively short period of time in order to apprehend one serious criminal. I would think almost any other form of police work would involve a much higher level of inconvenience to the public for the same level of success. For comparison of order of magnitude, look at neutron-activation bomb detectors, to be installed in airports. They are said to have something like a 3% false positive rate (and that is in controlled tests, so the real rate will most likely be higher). I don't have a number at hand for the actual rate of bombs in checked luggage, but let's say that it is 1 in 30 million (I'm sure it is less). That corresponds to 1 million false positives for every true positive. *That* is a high rate of error. And our society has chosen to spend nearly a billion dollars on that system. -- David desJardins, IDA/CRD ------------------------------ Date: Fri, 10 Nov 89 23:21:42 PST From: Eugene Miya Subject: Reference on the early history of Ada -- killing reliably I finally had a reason to dig thru the same box which had the following reference in the history of Ada. The quote comes from Pascal News #13, December 1978, then published by the Pascal User Group. This is a bound journal (although basically Xerox(tm) copied sheets); it was unrefereed. By way on context, it should be recalled that the initial proposals for the new DOD language (then refered as DOD-1, various "colors," and the progression: Strawman, Woodenman, Tinman, Ironman, and then Steelman) had 17 proposals based on Pascal (the last was based on PL/1, you can guess who proposed that 8). So, the Pascal community was thrust into the limelight. (It was still a somewhat obscure language at the time.) The quote is authored by Andy Mickel, U MN, then the editor: Latest News About DOD-1 (ADA or DOD0) --Andy Mickel As we've told you in previous issues of Pascal News, the U.S. Department of Defense (DOD) has endeavored to procure a common programming language based on Pascal for all "embedded" computer applications -- computer systems attached to weaponry. Reliable software should kill people reliably! ..... [above description removed] ...colors: BLUE-Softech, GREEN-Honeywell Bull; RED-Intermetrics; and YELLOW-SRI International. The article goes on for two more paragraphs, but Andy's comment about killing was not taken lightly. Pages 9-10. So much for the history of programming languages. --eugene miya, NASA Ames Res. Ctr. formerly joint ANSI X3J9/IEEE P770 Pascal Language Standards Committee ------------------------------ End of RISKS-FORUM Digest 9.41 ************************