26-Feb-87 23:49:06-PST,28880;000000000000 Mail-From: NEUMANN created at 26-Feb-87 23:47:23 Date: Thu 26 Feb 87 23:47:23-PST From: RISKS FORUM (Peter G. Neumann -- Coordinator) Subject: RISKS DIGEST 4.52 Sender: NEUMANN@CSL.SRI.COM To: RISKS-LIST@CSL.SRI.COM RISKS-LIST: RISKS-FORUM Digest Thursday, 26 February 1987 Volume 4 : Issue 52 FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: B-1 plagued by problems (PGN) Computer loses bus (Mark Biggar) Human errors (Brian Randell) Possessed terminal? (pom) Entertainment risks (Walt Thode) Automatic Call Tracing for Emergency Services (James Roche, Charley Wingate) "Active" car suspensions (Graeme Dixon) Altitude-Detecting Radar (Matthew Machlis) Re: Results of a recent security review (Andrew Klossner) Re: Sherizen talk; auto-landing (Eugene Miya) Air Traffic Control, Auto-Land (Scott E. Preece) Risks of autopilots (and risks of solutions) (Bill Janssen) Another difference between electronic control in cars and fighters (Brent Chapman) Re: Hurricane Iwa (Scott Dorsey) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, nonrepetitious. Diversity is welcome. (Contributions to RISKS@CSL.SRI.COM, Requests to RISKS-Request@CSL.SRI.COM) (Back issues Vol i Issue j available in CSL.SRI.COM:RISKS-i.j. MAXj: Summary Contents Vol 1: RISKS-1.46; Vol 2: RISKS-2.57; Vol 3: RISKS-3.92.) ---------------------------------------------------------------------- Date: Thu 26 Feb 87 21:12:09-PST From: Peter G. Neumann Subject: B-1 plagued by problems To: RISKS@CSL.SRI.COM (From the Stanford Daily, 26 Feb 87, in the "Dateline" section, compiled from the wires of the AP and the LA Times/Washington Post News Service) WASHINGTON -- Government investigators said Wednesday that as many as half of the new B-1 bombers at a Texas air base have been grounded in recent weeks because of nagging technical problems and that the aircraft's shortcomings may persist well into the next decade, contrary to public statements by the Air Force. During hearings before subcommittees of the House Armed Services Committee, Chairman Les Aspin, D-Wis, said the bomber's heart -- its defensive electronics system -- not only fails to jam enemy radar signals but actually serves as a beacon illuminating the B-1 as a target. Government Accounting Office officials ... testified that the problems with the $28.3 billion bomber program, especially the critical defensive electronic countermeasures (ECM), are far more serious than Air Force officials have acknowledged. GAO officials also predicted that the Air Force will have to ask Congress for substantially more money in coming years to repair and upgrade the bomber. ------------------------------ Date: Thu, 26 Feb 87 10:58:17 pst From: markb%sdcrdcf.UUCP@JOVE.CAM.UNISYS.COM (Mark Biggar) To: CSL.SRI.COM!RISK@sdcjove.UUCP Subject: Computer loses bus The Los Angeles bus system (also known as the Rapid Transit District (RTD)) uses a computer to keep track of its buses. The computer knows which bus is traveling which route at what starting time. The computer also has the complete time schedule information. The computer can be used to estimate the position of any bus using this information. On Feb. 25 the driver in trouble radio alarm was set off on bus #181, the computer was asked where the bus was and the LAPD was notified. The LAPD patrol unit that responded to the call could not find the bus, so they called in more units. They still could not find the bus and asked for a helicopter to help search for it. After about a hour, the bus driver was located in the drivers' lounge at the bus yard. The bus was in the repair yard and the repair crew had accidentally set of the alarm. It turned out that the driver had assumed that the repair yard had told the RTD computer that the bus was out of service, and the repair yard thought that the driver had told it. Mark Biggar Unisys - System Development Group, Santa Monica {allegra,burdvax,cbosgd,hplabs,ihnp4,akgua,sdcsvax}!sdcrdcf!markb markb%sdcrdcf@CAM.UNISYS.COM ------------------------------ From: Brian Randell Date: Thu, 26 Feb 87 19:11:49 gmt To: RISKS@csl.sri.com Subject: Human errors There was a very interesting documentary on BBC TV in their QED series here last night, entitled "A Fall from Grace: Patterns of Human Error", which contained quite a bit of material of relevance to RISKS. The programme (Yes, that is how even I spell it when it isn't intended for a computer!) used as its principal illustrations the 1977 collision of two Jumbo jets at Tenerife airport, and the task on making tea! Various types of human error were described, and discussed with several experts, including Professor Jim Reason, (Dept of Psychology, Univ. of Manchester), Dr. Ivan Brown (Applied Psychology unit, Medical Research Council), and David Embrie (sp?), an ergonomist from Aston University. The principal thing which I learnt, to my shame, from the programme was that psychologists seem have done a lot of useful study of the many different types of errors that even highly trained human beings make when exercising a sophisticated skill. Some comments I jotted down: (1) One could learn much of relevance regarding the errors made in carrying out highly skilled safety-critical tasks, such as piloting an airplane, or in a nuclear control room, from studying the errors made in inconsequential tasks (hence the tea making example, which when you think about it, does involve considerable, albeit informal, training) - i.e., the underlying causes seem to be similar, even if the consequences of errors are grossly different. (2) With a highly skilled activity, you make more mistakes if you do it consciously. This particularly applied to "sequencing" errors, such as missing or repeating a step. For example, if you are so following a well-known sequence of actions, on mental auto-pilot, and then suddenly become aware of your actions, there is a good chance of your resuming the sequence at the wrong place. (3) When you have learnt two similar sequences, you have, so to speak, constructed two similar competing "action daemons" - one can acccidentally switch to the wrong one. This was illustrated with an account of how one of the pilots (who was very skilled, and spent much time training others) was thought to have reverted to a pattern of actions which he was familiar with from simulator training, which did not quite match reality in the way that the pilot was supposed to communicate with the air traffic controller. (4) One characteristic of error-proneness concerns the notion of "field dependence" - some people have difficulty, and are slow, at picking out a relevant object from a complex field of view - a sort of mental tunnel vision, for which there are standard tests. Pilot training would probably select such people out, but drivers might well suffer from this, and the idea of using the standard tests to decide whether someone should have a driving licence was unlikely to be acceptable. The programme also contained a well-illustrated, though to me rather more expectable, account of the problems of designing interfaces to try to minimise human error - mainly illustrated by control room design, with reference to Three mile Island. Today I telephoned Prof Reason, and had a very interesting chat with him. We have arranged that he will come and give a talk to our Systems Research Group, and I have been given the following interesting sounding reference: New Technology and Human Error (ed. J. Rasmussen, K. Duncan, & J. Leplat), Wiley 1983, to which he contributed several chapters. My hope is that his ideas on error classification might be of relevance to the sorts of problems that s/w (and h/w) engineers suffer from which result in residual design errors in complex computer systems. My apologies to readers for whom all this is familiar - perhaps I should have taken Psychology 1, after all! Brian Randell - Computing Laboratory, University of Newcastle upon Tyne ARPA : brian%cheviot.newcastle.ac.uk@cs.ucl.ac.uk UUCP : !ukc!cheviot!brian JANET : brian@uk.ac.newcastle.cheviot ------------------------------ Date: Thu, 26 Feb 87 09:48:41 PST From: pom%under.s1.gov@mordor.s1.gov To: RISKS@csl.sri.com Subject: Possessed terminal? Since WWN is usually quite authentic, I will entertain some speculation on the topic. While 'electric currents' cannot be ruled out (an incompetent electrician could put full voltage into the 'ground' and many countries use 220V rather then US style 110V), the most likely explanation seems be the good old 'VDT stress'. (VDT = Video Display Terminal). There is a big volume of writing on the topic and even some solid information. Radiation (soft x-rays from CRT) was often blamed but informed consensus (which agrees well with my own observations) is that stress is psychological. Introduction of any 'computerised system' could be an enormous trauma to people who were never exposed to the computers (even when all you do is replace IBM Selectrics with the word processors <=:: I have seen secretaries crying and thinking of quitting or even retiring from the workforce for good). The proper procedure for converting to computer system is as follow: 1) Introduce terminals to the workplace, while doing the 'real work' with the old, manual system. 2) Put some games on the machine and let people play with VDTs (perhaps after hours or during lunch breaks). 3) Introduce e-mail, first just as alternative to phone call or memo, so that it is not NEEDED to get the job done. 4) When everybody (as measured by volume of use) is comfortable with the system, put some work-functions on the new system. 5) After a month or two, convert the rest. (You may find out that some people will quit or ask for a transfer, even with slow transition; those requests for transfer should be honored from the start.) I wonder how may 'mysterious accidents' that occur after new 'sophisticated safety systems' (e.g. in nuclear power plants) are introduced are caused by ignoring these simple common sense rules. pom ------------------------------ Date: 26 February 1987 0736-PST (Thursday) From: thode@nprdc.arpa To: risks@csl.sri.com Subject: Entertainment risks I generally favor the broad interpretation of what gets into this list. In that spirit, I offer the following item from the San Diego Evening Tribune of Feb. 25. It may or may not be "computer risk" related: "Los Angeles (AP) - Dialing a telephone is sometimes a gamble, as callers found out when they got "Dial-Porn" instead of state lottery information because of a switched line. "Pacific Bell fixed the problem yesterday, but before that callers heard a suggestive recorded message from a sultry-voiced woman when they sought Saturday's winning lottery numbers. "Maria de Marco, who manages 976 prefix lines for Pacific Bell, said it wasn't known whether the switch was a prank or an accident..." [Since most telephone systems are now extensively computer controlled, this certainly falls into the class of human misuse of computers. PGN] In the same paper there was another item, also datelined Los Angeles, that described the confusion of some Lawrence Welk compact disk buyers when their mislabeled and mispackaged CDs turned out to contain the soundtrack from a movie about former Sex Pistols member Sid Vicious. [I decided not to delete this paragraph on technology-irrelevance grounds. It could have been a computer-related problem! PGN] If a computer is involved in these instances, it would appear to be one with a sense of humor. --Walt Thode (thode@NPRDC) [Even if one wasn't involved, it has a sense of humor! PGN] ------------------------------ Date: Wed, 25 Feb 87 10:29:45 est From: James Roche Subject: Re: Automatic Call Tracing for Emergency Services Apparently-To: risks@sri-csl.arpa [...] As a firefighter in Monroe County (where Rochester is located) I can offer some insight to the troubles of the 911 system here. The 911 dispatch center here provides services for more than 80 county-wide emergency agencies (police, fire, ambulance). That is reportedly more than any 911 center in the US. Among the problems encountered are that fire district boundaries don't match postal service boundries which don't match ambulance service boundries which don't match town boundries, etc. Therefore when the ALI indicates a particular address is in Town X is is necessary for the dispatcher to turn to another screen and determine which police/fire/ambulance agencies are to be dispatched. Other problems encountered with 911 include the fact that the entire county is served by more that one phone company. Most of the county is served by Rochester Telephone which has set up its computers to route all Monroe County 911 calls to the 911 dispatch center. There are however locations in the county which are served by New York Telephone. NYT has set up its computers to route the 911 calls from Monroe County to the Syracuse dispatch center (70 miles east). The dispatcher on the Syracuse end must recognize the call is from Monroe County and route the call to the Monroe 911 center. There are also areas of the county served by Ogden Telephone. I don't know how they handle the 911 calls. >(Incidentally, the county Commissioner of Public Safety took this >occasion to complain about duplicate street names within the county ... While it is not clear that eliminating duplicate street names would have avoided the above problem, it would eliminate other problems. Not all emergency calls received by the 911 dispatch center come in via the 911 number. Many calls are still received on the old 7 digit number. When a call comes in on that number the pertinent data for the address is not displayed. The dispatcher must then determine which one of the many duplicates the caller is referring to. I recall hearing 6 fire departments dispatched one day to a false alarm on East Avenue because there are multiple East Avenues within the county. The call was received on the 7 digit number and the caller gave incomplete information to the dispatcher (intentionally I imagine). The county feels that it must continue to provide service on the 7 digit number since for many years phone stickers were distributed with that 7 digit number. Also the residents the the areas served by New York Tel are encouraged to use the 7 digit number to avoid delays by going through Syracuse. Jim Roche UUCP: rochester!roche University of Rochester Computer Science Department Rochester, NY 14627 ------------------------------ Date: Thu, 26 Feb 87 23:44:03 EST From: Charley Wingate To: risks@csl.sri.com Subject: Re: Automatic Call Tracing and Addresses Here in Howard Co. Md., the county government took a big step years ago and renumbered all the addresses so that with in some quanta the street numbers are not only unique, but they also give the physical location of the property. This has done wonders for getting the FD to the right place. Unfortunately... "Laurel" phone exchanges lie in four counties; Laurel zip codes in three. This makes dialing 911 a bit of an adventure because you had better know which county you are in. Sometimes even this doesn't help. One zip code was believed by the counties to lie entirely in P.G. county, when in fact a small piece of it lay in Montgomery County. This meant that these people got no county services-- no fire, no trash, nothing. After years of bickering, the Postal Service cut the gordian knot and created a new zip code just for these people. The moral: "Garbage in, Gospel out" doesn't just apply to computers; they can "bless" information that never came near them! C. G. Wingate U of Maryland, Dept. of Computer Science, Coll. Pk., MD 20742 ------------------------------ From: Graeme Dixon Date: Wed, 25 Feb 87 19:14:57 GMT To: risks@csl.sri.com Subject: "Active" car suspensions Since the discussion has once again come around to the use of computers in cars the "... most important single automotive advance since the accelerator pedal ..." may be of interest. There have been a number of articles in British motoring magazines (Car Oct 86, Fast Lane Jan 87) over the last few months describing the Lotus "Active" suspension. This consists of a replacement for the normal passive suspension of dampers, springs, and anti-roll bar, by a sensing system, computer, and a set of hydraulically controlled actuators. The sensors return the cars relative movement and driver inputs, and the computer adjusts the actuators to compensate. The resulting handling characteristics are by all accounts superb - no roll, no understeer, no oversteer, just perfectly balance handling. Various parameters used by the computer may be adjusted to provide different levels of ride, prompting one of the writers to speculate that it would "be possible to build a schizophrenic car with His and Hers alternative handling at the flick of a dashboard switch." One of the more contentious claims of the system is that "it is truly fail-safe". By providing a "get-you-home stand-by suspension" computer failure does not render the car unusable. One of the articles even describes the cars behaviour when the system is "dumped" as the car is negotiating a corner - the car switches suddenly from neutral handling to oversteer prompting the driver to think one of the rear tyres had punctured. What they didn't try was the effects of over compensation though! It will be a few years before active suspensions appear in cars (Lotus are intending to use it in their supercar the Etna which they are currently developing), but given that Lotus have been recently bought by GM, and a number of rivals (notably Mercedes-Benz) are developing similar systems, then this should provide another fertile area for discussion when the time comes.... Graeme Dixon ------------------------------ From: Date: Wed, 25 Feb 87 16:10:34 EST To: risks@csl.sri.com Subject: Altitude-Detecting Radar It is true that Mode C capability costs a bit of money, but I think the majority of people who own planes could afford the extra $1500 or so, especially considering the added safety. As to 3-D radar, it would be very nice but I am under the impression that it is quite impossible, realistically speaking, with the present technology. A professor here at MIT who flew for the Navy for 20 years told me it is reasonable to make altitude-detecting RADAR, but that it is only economically reasonable for tracking a single target at a time. Aircraft such as the F-14 and F-16 can track several targets at once, but those systems are very expensive and have MTBF averages of only several hours of operation because of their complexity. ------------------------------ Date: Wed, 25 Feb 87 12:59:02 PST From: Andrew Klossner To: RISKS@CSL.SRI.COM, HOLSTEGE@SUSHI.STANFORD.EDU Subject: Re: Results of a recent security review "Fifth problem: A program can be created with "OWNDIR" privileges. While it is running, it has all the privileges associated with the account on which it resides." Interesting ... did they license the use of this invention from AT&T, the patent holder? -=- Andrew Klossner (decvax!tektronix!tekecs!andrew) [UUCP] (tekecs!andrew.tektronix@csnet-relay) [ARPA] Tektronix, Inc., Wilsonville, OR [... and will someone sue AT&T if, after a license is duly obtained, a devastating Trojan horse is perpetrated using this flaw/feature ? PGN] ------------------------------ Date: Thu, 26 Feb 87 16:23:03 PST From: Eugene Miya To: risks@csl.sri.com Subject: Re: Sherizen talk; auto-landing I think an apology is in order. I sent my notes to the CPSR Sherizen talk to Peter (not with the intention of posting to the net). Locally, we are trying to have discussions on security trying to forego problems of discussing security both when it was tried in unix-wizards (and it subsequent list) and info-vax (for the VMS side). Although the Sherizen meeting of CPSR was open, our other meetings are not (they are not classified either). Regarding auto-land: I don't know if I would trust such a system yet. I know few pilots who would not feel at least a little uncomfortable. Actually, I think systems like this would be great Darwinian tests of AI. The posting implied we control everything. This is not true. The plane is not everything, there are other planes and obstacles out there. Put the developer on the plane, let his or her system land the plane. If the plane survives, the developer goes on to create their next system. (Might not be enough, but a good first cut.) Similar tests for things like MYCIN, etc. can be used (infect using a blood disease, developer then must trust system for diagnosis ;-). Sound a little too real world? We know less about the real world than many think. Thinking is not enough. --eugene miya [In the past I have been extraordinarily careful about not including obviously personal messages without explicit permission. In this case I clearly goofed. The message somehow seemed to be of general interest and addressed to a large list... And it was getting late. Sorry, Eugene... PGN] ------------------------------ Date: Wed, 25 Feb 87 09:13:49 CST From: preece%mycroft@gswd-vms.ARPA (Scott E. Preece) To: RISKS@csl.sri.com Subject: Air Traffic Control, Auto-Land Use of automated landing also would leave the crew more free to spend its time looking for things out of the ordinary -- unreported traffic, patterns of air movement, the effect of the wind on preceding traffic, the overall condition of the aircraft -- that automated systems are not good at detecting. scott preece, gould/csd - urbana, uucp: ihnp4!uiucdcs!ccvaxa!preece ------------------------------ Date: Wed, 25 Feb 87 17:02:01 CST From: Bill Janssen To: RISKS@CSL.SRI.COM Subject: Risks of autopilots (and risks of solutions) In Risks Digest 4.51, Matthew Machlis questions whether there may be risks of pilots losing their flying skills, due to flying for extended periods on autopilot. At a conference last year, I spoke to folks from a major commercial aircraft manufacturer, who were concerned about the same thing. (One of the speculations about KAL 007 was that the pilots just `lost track' of what they were doing.) This firm had the thought of dividing the cockpit in two, using one half for flying the real airplane, and the other half for a training simulator. The pilots would trade off acting as `system monitor' and practicing `real' problem flying. The problem with this solution was loss of orientation, along the lines of "Oh, damn, I just put the plane in an unrecoverable spin; well, restart... that's funny, nothing seems to happen... Ohmygod, I'm sitting on the *real* side". Bill ------------------------------ To: risks@csl.sri.com Subject: Another difference between electronic control in cars and fighters Date: Thu, 26 Feb 87 17:03:14 PST From: Brent Chapman Another key difference, which to me seems just as important as the maintenance issues already mentioned, is that cars (generally!) aren't fitted with ejection seats. A driver can't punch out when things get weird. Also, cars tend to be operated in much more crowded conditions. Usually in fighters (except possibly during takeoff and landing), you really don't have to worry about what your plane will come crashing down on, because most operations (both real and training) occur over very sparse areas. In a runaway car, on the other hand, you stand a significant chance of wreaking considerable havoc among other vehicles travelling in your vicinity, as well as bystanders and property near the roadway. Brent ------------------------------ Date: Thu, 26 Feb 87 12:24:31 est From: Scott Dorsey To: RISKS@CSL.SRI.COM Subject: Re: Hurricane Iwa (RISKS DIGEST 4.51) Winds from Hurricane Iwa passed through a small mountain pass, gathered pressure from the narrow slit, and knocked out power lines which carried power to most of Central Oahu. They also did serious damage to an army base on the exiting winds side of the pass, opening warehouses filled with emergency supplies like sardine cans, or ripping the prefabricated buildings away from their foundations while leaving the contents sitting. The base was without power for three weeks, and without water for about two. The Mayor of Honolulu asked the military for help, and they refused (being much harder hit than the civilian community, mainly due to the damage at this base). There were several scathing editorials in the Advertiser, but the military did not really release any information about the extent of the damage. The island of Kauai was worst hit. Although the generating system was not heavily damaged, there was no way to restart the generators without power, as no one had foreseen that all the turbines would go down at once. The Navy sent a nuclear submarine from Pearl Harbor over to Kauai to provide power for the starters, but by the time it arrived, the engineers had restarted the system, using almost a hundred automotive batteries. > In the afternoon, winds started rising, and the Weather Service issued a > Hurricane Watch, then quickly a Warning, but still didn't have a precise fix > on Iwa, nor accurate information on speed or direction. At about noon, state employees were sent home, schools were cancelled. I was in downtown Honolulu at 3:00 or so. All the shop windows were taped up, and a cold, dry breeze blew through the streets, picking up bits of paper and carring them around. There was not another soul on the streets, and I was not able to get back to the base, as all the buses had stopped. I eventually got someone to come down and pick me up, and we were the only car on the roads. I don't know much about the damage to Honolulu, being stuck on base for a while because I had no form of transportation (tree fell on car). > [This could be a separate story in itself, but suffice it to say that the > Civil Defense Emergency Broadcast system didn't work. Besides all the TV > stations, all the radio stations---except one--- went off the air that > night. The single radio station that had an operating emergency generator > was running "on automatic", playing religious music.] Nope. Radio station KGU was on almost all the time, on their standby generator. They were off for a few hours when their antenna was damaged, but brought the transmitter (at the studio site) back up with a long wire dipole. At first they were calling various authorities, but after the phone went out, they just sat around and played music, complaining about the weather. I don't think that the extent of the damage to the military installations was ever revealed, so you can probably say you saw it first here. It doesn't have much to do with risks from computer systems, but it does have a bit to do with risks to computer systems, as well as anything else that uses electricity. At least, I know my PDP-11 did go down at the time. Scott Dorsey Kaptain_Kludge ICS Programming Lab, Rich 110, Georgia Institute of Technology, Box 36681, Atlanta, Georgia 30332 ...!{akgua,allegra,amd,hplabs,ihnp4,seismo,ut-ngp}!gatech!gitpyr!kludge ------------------------------ End of RISKS-FORUM Digest ************************ -------