6-Mar-87 14:00:10-PST,19340;000000000000 Mail-From: NEUMANN created at 6-Mar-87 13:58:51 Date: Fri 6 Mar 87 13:58:51-PST From: Peter G. Neumann Subject: RISKS DIGEST 4.57 To: RISKS-LIST@CSL.SRI.COM RISKS-LIST: RISKS-FORUM Digest Friday, 6 March 1987 Volume 4 : Issue 57 FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: [SOME MORE NEW THOUGHTS (?) ON OLD BUSINESS!] Re: Air Traffic Control, Auto-Land (David Redell) 911, drive-fly by wire, risks, and the American work ethic (Wes Williams) Re: drive by wire (Bennett Todd) Autoland (Peter Ladkin) Re: Puget Sound Ferry Boats (Bjorn Freeman-Benson) Credit Card Limits (Clive Dawson) NSA Monitored McFarlane House, Magazine Reports (Don Hopkins) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, nonrepetitious. Diversity is welcome. (Contributions to RISKS@CSL.SRI.COM, Requests to RISKS-Request@CSL.SRI.COM) (Back issues Vol i Issue j available in CSL.SRI.COM:RISKS-i.j. MAXj: Summary Contents Vol 1: RISKS-1.46; Vol 2: RISKS-2.57; Vol 3: RISKS-3.92.) [NOTE: SRI WILL HAVE A WEEKEND POWER OUTAGE, AND THUS THIS ISSUE IS COMING OUT EARLIER IN THE DAY THAN USUAL. (If this issue does not get far enough through the mailer queue before the outage, some of you may get two copies...)] ---------------------------------------------------------------------- Date: Fri, 6 Mar 87 13:20:07 PST From: redell@src.DEC.COM (David Redell) To: RISKS@csl.sri.com Subject: Re: Air Traffic Control, Auto-Land Recent discussions have compared risks of computerized autolanding of planes to those of computerized launch-on-warning of nuclear weapons. I think lumping these together can be misleading. For example, as Mr. Shapir points out: > ...the question should not be whether automatic systems will cause > accidents, but whether the accidents' cost would be greater or smaller > than the cost of accidents in the human systems they replace. This is A good question, but not THE only good question. In cases where an existing situation is being automated, I agree that this is the right question to ask. Often, however, the prospect of using high-speed computer control is cited in support of plans to establish new situations where human control would be unworkable. Subsequent discussion often focuses on the relative risks of human vs computer control. But if neither works, then the mistake is to get into the situation in the first place! Ideas like computerized launch-on-warning or AI-based weapons release for SDI are not bad ideas because humans could do those jobs better -- they are bad ideas because we are moving toward situations where neither humans, nor computers, nor any combination of the two can be trusted to do the right thing in the time available. One of our responsibilities as professionals is to try to identify and call attention to such situations before the choice degenerates to one of arguing about which of several unworkable options is the least unworkable. Dave Redell ------------------------------ From: eww@OBERON.LCS.MIT.EDU (Wes Williams) Date: 6 Mar 1987 1516-EST (Friday) To: RISKS@CSL.SRI.COM Subject: 911, drive-fly by wire, risks, and the American work ethic (interrelated thoughts) 911: Having been associated with the Emergency Services for some 20 years, I do not find the 911 articles surprising. I remember the horror stories from the times of conversion from "local" operators to those of the more regional type. People were accustomed to picking up the phone and yelling help or fire and screaming the address to the operator. While in the "new" system, the "0" DIALED in the phone would connect you to the local operator (usually) within the town or city of origin. Here the most severe complications were duplicate street names or same names suffixed by St. or Terr. or Place or Circle. As time went by the switchboards dissapeared from the local towards the regional type. Now the problems grew to the kind of identifying the neighboring community possibility. Here the operator would be the one in the position of determining the locality of origin of the call, as well as the correct address. Sometimes (1960's era to present) multiple community dispatches were heard for the same address in different municipalities. The problems have not been rectified, only compounded by the advent of differing phone systems and overlays of telephone exchanges. Software may or may not be the problem, as the best software can only rely on input (electronic or manual). As area codes are becoming more and more prevalent, it may be necessary to soon dial an area code to report the fire across the street. hmmmmmm..... Point 1. System modification (hard or soft) is not always the answer unless the root problem is solved. Even here, there will forever be unresolved complications. Example: a non-English speaking (obscure language) person will call an English speaking relative in another town (or state) to report an emergency. Second party calls are always the hardest to handle. The time is not yet at hand to convert the emergency services to AI ! Steer/Drive by wire: These discussions are relevant to Risks as they are or will be implemented at some time. BUT! It is sort of the same as adding the computer to a small business; there are times that it is just not appropriate. Mechanical design considerations have been for some time at the technological point to eliminate any of the problems (reasons) for such a computer system. Ask a race car driver what computer systems he wishes. Here the answer seems to be more emergency condition indicated than technologically capable. That driver wants a system to turn on the fire extinguishment system in .000001 second of the explosion or fire, and yet you will not see the air bag pop out of the MECHANICAL steering wheel. You have seen the severe crashes these people are exposed to, and yet they want the machine to be at hand, not computer. This will hold true unless the people start loosing to such a system, thus proving its merit. Point 2. This is the, "eliminate the man" syndrome. If the speed and complexity of the systems are such so that a computer insertion to control it is necessary, then it is time to consider removal of the human element. This bridge is a hard one to cross. Project loss due to failure and the price of backup systems put the cost of such projects over the top. We still put the wo/man above price and yet when a multibillion dollar project is launched, the requirement of the human to be onboard is still paramount. Protection of the systems, uncalculated emergency procedures, patches and repairs incapable of the onboard systems are only feasible with the HANDS and brains of the crew, supported by their electronic and human counterparts in remote. Major system failure will cost not only the project, but also the crew. This possibly is the impetus for quality in design and manufacture. Do you work more carefully when there is a human life in the balance at the reception of your output? i.e., The program writer who discovered his program was inside the operating room during a heart transplant, and had a few thoughts about the possible bug. Work ethics in the U.S.: Systems installation into the chain of mechanical elements is obviously an expected outgrowth of our technology. The desire to have modern systems replacing 100 year old mechanical ones runs back as far as the fellow that removed the square corners from the wheel. The real question is if the can opener really needs that keyboard input in conjunction with the clock card in order to do the job. If it is a desire of the customer to have such a system, so be it. System implementation seems more of, "Gee, look what I made. Where shall I put it?", than here is the problem, what shall we do to make it better. Total redesign may be more appropriate than added-on systems. It is up to us to say enough is enough and initiate that type of improvement rather than amend a system. >From: sigma!roman@entropy.ms.washington.edu (Bill Roman) >*RUMOR* >I can't vouch for this personally... but a few years ago I spoke to a >contractor who said he had been approached to write software for the >Issaquah class ferries. [...] My friend refused the contract. This type of reaction to an idiotic set of circumstances is of the highest quality. The only neglect here (not mentioned) was a blast to the authorities requesting the work. It is a shame that in order to keep position in relation to other professionals, one must remain mute on problems such as this. I wonder how many lines of code be eliminated or dollars saved (redundant?) if there were a majority of professionals that acted in this manner? Tell me, are the Risks that we are seeing more of a moral question or one of simple incompetence? eww@oberon.lcs.mit.edu Wes Williams ------------------------------ From: Date: Fri, 6 Mar 87 06:03:40 est To: ecsvax!CSL.SRI.COM!RISKS@mcnc.org Subject: Re: drive by wire Representatives of GM recently gave a presentation here at Duke on the Chevrolet Corvette Indy. This "show concept car" (a one-of-a-kind) has about everything on it people have been worrying about in this forum; I went to the presentation and nagged the engineers about the points of concern that have been raised here. This car was built by Lotus Cars Ltd.; it might have been the project that started this discussion. For starters, the term "drive-by-wire" is used in their glossies *not* to refer to the computer controlled steering, but to computer controlled throttle! The car is four-wheel-drive, with computer control over the split of torque between the front and rear wheels, designed to maintain maximum traction in all conditions of acceleration/deceleration. The "gas pedal" is connected to a sensor (and has a hydraulic ram behind it so the computer can simulate the feel of a mechanical linkage); the sensor concludes what acceleration the driver wants and delivers torque to the front and rear wheels. This is probably the most RISKy part of the whole car, in my humble opinion. It is a bit more comprehensive than the computer controlled idle adjustments and suchlike that are getting to be common these days. It also has a computer controlled four wheel active suspension; when I asked them about the failure modes and potential RISKs in this subsystem, they replied that in the event of loss of power to the hydraulic system driving the active suspension, the coil spirings hold the car at its normal height above the wheels, and the hydraulic rams are designed to fail under loss of power into reasonable shocks. The ride would be mushy, but not dangerous (unless of course it failed in the bottom of a really tight turn). The computer controlling the system (1) has internal sanity checks throughout, and (2) has multiply redundant sensors; whenever any inconsistency is found in the system it fails into the powered down mode. Finally, the computer controlled steering. The front wheels are normal manual rack-and-pinion steering; the front steering linkage has a sensor on it so that the computer can tell how far you have the front wheels deflected. Based on the deflection of the front wheels, the speed you are going, current acceleration vector, "weight" currently on each wheel, and suchlike, the computer deflects the rear wheels. In particular, at low speeds, the rear wheels turn the opposite direction from the front, tightening the turning radius substantially. At high speeds, they turn the same direction as the front wheels, making fast lane changes smoother; instead of slewing around, and rocking from side to side, the car tends to slip crabwise laterally. The total deflection available to the rear wheels in the prototype is 20 degrees left or right of center; according to one of the engineers there they only would leave 5 degrees available in a production system (that's all that is needed). The system is once again equipped with multiple internal sanity tests, and dumps at the first sign of trouble; large springs center the rear wheels if the system dumps. In tests where they deliberately cause the critter to fail turned as sharply as possible, they found that at slow speeds the car could be stopped safely, and at high speeds the driver could keep control by steering the front to compensate (and proceeding slightly angled down the road). All in all, the severity of symptoms seem much less severe than a blowout; if the likelihood of such a failure can be reduced as low, then the steering shouldn't introduce too much RISK. Bennett Todd, Duke User Services, Durham, NC 27706-7756; +1 919 684 3695 UUCP: ...{philabs,akgua,decvax,ihnp4}!mcnc!ecsvax!dukeac!bet BITNET: DBTODD@TUCC ------------------------------ Date: Fri, 6 Mar 87 13:03:07 pst From: ladkin@kestrel.ARPA (Peter Ladkin) To: risks-request@csl.sri.com Subject: Autoland Those who do not like category IIIA autoland (auto up to main wheels on the ground, pilot has to lower the nosewheel) might avoid flying the Concorde, which uses it routinely at Kennedy and London Heathrow, and might also avoid flying in to London Heathrow, which I understand has Cat IIIA on all runways, used routinely in English Weather. It's been thoroughly tested in the field for many years. peter ladkin ------------------------------ Date: Fri, 6 Mar 87 07:56:53 PST From: bnfb@beaver.cs.washington.edu (Bjorn Freeman-Benson) To: RISKS@CSL.SRI.COM Subject: Re: Puget Sound Ferry Boats From a Puget Sounder who has followed the story in the papers... The computers for the Issaquah class ferries were built by a private contractor to MP&E. This private contractor turned out to be a one man shop who did little or no quality control and went belly-up after the ferries were built. He/she did not leave any documentation behind. The results were: (a) The computers are poorly designed and built -- at one point the boards physically fell out of the card cage while under way. (b) With no documentation, repair would be incredibly expensive. (c) The failure of the computers (starting with the maiden voyage) had caused the public to mistrust them, and so replacement by a physical system is occurring. (d) Many of the failures have been attributed to physical parts such as small relays. (i.e. The software said "slow down" but engine didn't.) A better overall system design would have helped. Bjorn N. Freeman-Benson ------------------------------ Date: Fri 6 Mar 87 15:12:47-CST From: Clive Dawson Subject: Credit Card Limits To: risks@CSL.SRI.COM [This is another instance of an old problem, but worth rehearing.] Yesterday I received a nasty letter from my credit union stating that I had exceeded my VISA card's authorized credit limit of $500 by $203. They advised me to pay up immediately or face the consequences, etc. etc. This was a bit of a surprise, considering that my credit limit was actually $2000. The very next letter in my stack of mail contained the following: Dear Member: Please accept our apology for the recent letter stating you were over your credit line. We were attempting to implement a credit line increase into the system. Due to a programming error by our processor in Dallas, the old credit line was inadvertently removed and only the increase appeared on the account. Some members were declined on purchases due to this error. The new credit lines are now in the system and your account is in good standing. Your March statement will reflect the new credit line increase. We regret any inconvenience this may have caused you. Sincerely, [etc.] I guess I was one of the lucky ones who didn't even notice the problem until I received both letters simultaneously. I would not have been at all amused had I learned of this on an out-of-town trip trying to rent a car or something. Clive ------------------------------ Date: Fri, 6 Mar 87 13:07:58 EST From: Don Hopkins To: risks@csl.sri.com Subject: NSA Monitored McFarlane House, Magazine Reports [A few new items] Conversations Said Taped (United Press International) The government secretly monitored the home telephones of Robert C. McFarlane after he stepped down as President Reagan's national security advisor, according to an article in the Progressive magazine. The magazine article said a National Security Agency electronic device was found in the sewing closet of McFarlane's home in Bethesda in January during a sweep ordered by his attorneys. Spokesmen for the NSA and for McFarlane refused comment. The White House said it would have no comment until it saw the magazine, which is to be on newsstands Saturday. The magazine quoted intelligence sources as saying that phone conversations of senior U.S. officials have been recorded for "archival purposes by the Pentagon and the CIA and for communication security by the NSA." In the article entitled, "The White House Tapes, Again," the magazine quoted sources as saying the program produced "a still-undisclosed archive of recorded conversations" involving Reagan, Vice President Bush, former White House chief of staff Donald T. Regan and former National Security Council staff members Oliver L. North and John M. Poindexter. The article, written by freelance reporter Allan Nairn, said McFarlane, who left the White House in December 1985, had been falsely told that a security unit on his home phone had been deactivated. It said the unit uses a computerized encryption device that makes a call unintelligible to anyone trying to listen in without the proper equipment and authorized code. The article said that the monitoring of top officials generally seems to have been done on a basis of express or implied consent and therefore would not appear to violate federal communications laws. In McFarlane's case, however, the monitoring continued after he left the White House, the magazine said. A government team, according to the magazine, removed the unit's handset from McFarlane's home, but, unknown to McFarlane, left intact the system's control panel that enabled NSA to monitor calls, and in turn, record them. After leaving the national security adviser's job, McFarlane continued to have access to classified material as unpaid consultant until the Iran-contra affair was disclosed in November. He took a secret trip to Tehran last May in a fruitless effort to free American hostages in Lebanon. ------------------------------ End of RISKS-FORUM Digest ************************ -------