RISKS-LIST: RISKS-FORUM Digest Thursday 5 May 1988 Volume 6 : Issue 78 FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: Rambling robot disrupts evening news broadcast (Donn Seeley) Phone fraud -- $150,000 (PGN) Blame it on the computer -- lost homework! (PGN) Re: Creating alternatives to whistleblowing (Henry Spencer) KAL 007 (Robert Dorsett) Micros & Airlines - A New Angle (Anand Iyengar) Ollie North Helps PROFS sales (David A. Honig) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, nonrepetitious. Diversity is welcome. Contributions to RISKS@CSL.SRI.COM, Requests to RISKS-Request@CSL.SRI.COM. For Vol i issue j, ftp kl.sri.com, get stripe:risks-i.j ... . Volume summaries in (i, max j) = (1,46),(2,57),(3,92),(4,97),(5,85). ---------------------------------------------------------------------- Date: Wed, 4 May 88 22:02:10 MDT From: donn@cs.utah.edu (Donn Seeley) Subject: Rambling robot disrupts evening news broadcast New York Times, 2 May 88 Television / Peter Boyer AT NETWORKS, CHEAP IS CHIC, SO PLEASE PARDON THE ROBOTS One recent Saturday night, Connie Chung, the anchor of the weekend version of 'NBC Nightly News,' was reading an urgent story about the Middle East, when she began to disappear. The studio camera had inexplicably begun to move from its position, pushing Ms. Chung's image from the screen as it glided across the studio floor. Ms. Chung might have motioned to the cameraman, except there was no cameraman. The source of her distress was a robot, one of NBC's new self-operating cameras, that had apparently gotten a case of wanderlust. ... [details about cost-cutting at NBC News, replacement of human cameramen by three robots at a cost of 'less than $1 million together'] ... On that eventful Saturday night, Ms. Chung realized that she was moving out of the camera's frame as she read the Middle East story. She considered scooting her chair, which is on wheels, in pursuit of the robot camera. But she remembered that she was stationed on a platform, 'and if I did move, I might have fallen off,' she said. Finally the robot collided with the stage manager, ending its journey but not its mischief. Having stopped, the camera began to pan the anchor desk, turning its lens even farther from the anchorwoman. Ms. Chung tried to lean into the picture, managing to get about half her face into the frame before cutting away to a taped report. Ms. Chung said that, over all, she has no particular objection to the use of robots to help NBC's cost efficiency drive. Had she been asked on the night of her misadventure, however, her view might have been different. Before the broadcast, a computer that prints scripts for use in the Teleprompter chewed up and rearranged some of her prose. 'I was being killed by machinery that night,' she said. 'If you'd asked me that night how I felt about non humans, well, it wasn't very favorable.' ------------------------------ Date: Wed 4 May 88 19:22:33-PDT From: Peter G. Neumann Subject: Phone fraud -- $150,000 Two Corte Madera CA teenagers were arrested for using their personal computers to search through lines of numbers, seeking access to credit card and toll-free numbers. They apparently racked up $150,000 in illicit phone calls during a three-month period. Their victims included PacBell, MCI, GTE Sprint, Future Tech, and All Net. Authorities believe they were part of a Marin County telephone fraud network. [Source: SF Chronicle, 4 May 1988, p. A2] ------------------------------ Date: Wed 4 May 88 19:12:26-PDT From: Peter G. Neumann Subject: Blame it on the computer -- lost homework! MODERN TIMES: When you were a kid, did you ever tell the teacher ``My dog ate my homework?'' Update: Navy Lt. John Ratkovich, a student at Naval Postgrad in Monterey, tells me that when homework was called for the other day, Lt. Comdr. Al Jones said ``May DOS ate it.'' Right. His disc operating system erased it all, and would a commander tell a fib? [Herb Caen, SFChron 28Apr88] ------------------------------ Date: Wed, 4 May 88 22:41:34 EDT From: mnetor!utzoo!henry@uunet.UU.NET Subject: Re: Creating alternatives to whistleblowing [RISKS-6.65] > * If I see a problem, should I let it continue even though it's not > in my 'area of responsibility'? (This may seem like a non sequitur, but all will become clear...) A book that might interest Risks readers is T.N. Dupuy's "A Genius For War" (Prentice-Hall 1977). It's an investigation of how, for about a century, Germany consistently produced the world's best armies -- not just bigger, but significantly better, man for man. (Specifically, German armies fought as if they were about 20% larger than they really were, and they inflicted 50% more casualties than an equal number of other soldiers.) (Dupuy's book is actually an interesting example of simulation uncovering real-world surprises. He started looking into the subject when attempts at numerical simulation of WW2 battles could not be reconciled with real life unless a fudge factor was introduced to give the Germans an advantage. He notes that similar fudge factors can be found in commercial wargames, if you go looking for them.) His major conclusion was that individual German soldiers were no better than their opponents: Germany's advantage was better officers, produced not by birth but by superior training. One aspect of their training particularly stood out (we're now coming to the relevant part...): the traditional stereotype of Germans being obsessed with blind obedience was wrong, dead wrong, for the officer corps. In fact, German officers had it hammered into them repeatedly that they were responsible for getting results, not for following orders, and that obeying orders was *not* an excuse for fouling up. If they saw a problem developing, it was *their* responsibility to see that something was done about it, orders or no orders, chain of command or no chain of command. After the Franco-Prussian war, General Moltke inserted the following in a new training manual: "A favorable situation will never be exploited if commanders wait for orders. The highest commander and the youngest soldier must always be conscious of the fact that omission and inactivity are worse than resorting to the wrong expedient." Every German officer heard the story of the major, being reprimanded for fouling up, who tried to defend himself by pointing out that he was following orders and that orders from a superior officer were legally equivalent to orders from the King. Prince Frederick Charles, who was delivering the reprimand, replied: "His Majesty made you a major because he believed you would know when *not* to obey his orders." This was not apocryphal folklore; Moltke himself witnessed the incident, and saw to it that it was incorporated into officer training, to make it clear what the priorities were. The result was an army which -- other things being equal -- consistently performed better than any other army on Earth. "[This system] enabled men who individually lacked the qualities of a genius to perform institutionally in a manner that would provide results ordinarily achievable only by genius." (Before anyone objects that Germany lost both World Wars, note that there is wide consensus that this was not the Army's fault. In WW2 in particular, it came frighteningly close to winning -- against larger and better-equipped opponents -- despite extensive political meddling in its decisions and operations.) How many companies (for that matter, how many *armies*) tell their staff anything like that? How many get results like that? Henry Spencer @ U of Toronto Zoology {ihnp4,decvax,uunet!mnetor}!utzoo!henry ------------------------------ Date: Thu, 5 May 88 13:42:28 CDT From: mentat@huey.cc.utexas.edu (Robert Dorsett) Subject: KAL 007 Cc: padraig@astro.as.utexas.edu, steve@ames-aurora.arpa Every 747 I've seen uses an inertial navigation system manufactured by Delco Electronics, a subsidiary of General Motors. It's a fairly primitive unit, capable of storing a whopping 10 waypoints at a time. There are three units on the 747, plus an optional card reader. The INS's cost about $100,000 each. Software updates are actually firmware updates, and referenced by version num- ber, rather than date. Since operators must purchase upgrades, it's inevitable that many carriers are operating old, obsolete INS's--perfectly legally. Many carriers wait until a break-down before a board swap, then just swap the latest version (or the latest version their maintenance department has stockpiled). The multiple units are used for redundancy inflight, but coordinates can be entered in an "intermix" mode on the ground, to save time. Crew procedures call for cross-verification of waypoints by both the captain and first officer before or during taxi. Most third-world airlines do not use the card reader, even if it's installed. Many third-world airlines have poor or dubious administrative practices, and keeping the cards up to date (not to mention current copies on each airplane and compensating for theft or misplacement) is a bit of a task. So what is done is the waypoint coordinates are entered from a computerized flight plan. These flight plans are obtained from the airline's dispatch office, which in turn buys them from a service (forget the name). The flight plans indicate the airplane's longitude, latitude, fuel burn, magnetic heading, projected altitude, etc., for every waypoint. The elapsed time is also given beside the waypoints. Waypoints are referred to by both name (remember, over-water navigation is area navigation) and coordinates from the perspective of the paper flight plan and the charts. The INS, however, only refers to waypoints by coordinates, which can lead to misinterpretation if, for example, an LED element burns out or a number is simply misread. The flight plans start at "enroute climb" and ends at "entry" at the ATC system at the target airport. There are four copies of the flight plans, each one color-coded by a stripe down the left side. After the INS's are stabilized on the ground, the airplane position is entered. Then, the waypoints coordinates are entered. After takeoff, if a "direct" routing is obtained from ATC, the autopilot is slaved to the INS. The INS runs the show until it's time to add more waypoints. Optionally, a flight director display can be called on the attitude diplays to cross-check INS flight commands. Optimally, the pilots (captain and first officer) verify INS navigational information with the flight plan. They are expected to cross-check longitude and latitude and establish that the airplane's heading matches the projected heading. The role of the flight engineer is to make sure that fuel burn is within acceptable limits. By the end of the flight, the paper flight plans are heavily marked to indicate deviations from the ideal flight characteristics. In a perfect world, the massive sequence of errors that led to the destruction of the KAL flight would not have occurred. Even if the captain entered a wrong waypoint, it's inevitable that the mistake would be noted later on, either via cross-check of the headings or of the actual cross-check of longitude and latitude. The INS units also provide a multitude of information beyond merely aircraft position, such as ground speed, track, true course, etc, all of which can be used to verify other characteristics. However, when we look at other factors, the "off course" theory might gain more credibility. First, a long-documented trait of many oriental aircrews is the absolute assignment of command on the captain. The captain often does *all* takeoffs and landings, and, in general, has absolute authority on the ship. The first officer is discouraged from voicing his opinions, and, even if he does, such opinions can be (and often are) completely ignored. The flight engineer is almost a non-entity. There have been cases of first officers getting promoted to captain with 15,000 hours with absolutely minimal time manipulating the flight controls of the airplane. These behavioral characteristics have been addressed at a recent flight safety conference by the Flight Safety Foundation in Tokyo, and have been documented for at least 25 years, by sources within the airlines and Western safety observers. Second, if the captain (we presume the captain enters the coordinates in the INS at the beginning of the flight) entered a WRONG waypoint, it might not be picked up, especially if there was a rushed start and a fast taxi. For credibility's sake, we'll assume that there was one waypoint error. Third, KAL aircrews are not viewed in the best light by the rest of the flying community. We can assume that, although they meet professional standards, there are deficiencies in training and conduct--credible given the earlier 707 blunder into the Soviet Union and numerous safety and operational discrepancies. Now, for the worst-case scenario: we have a docile first officer. Captain screws up the entry of at least one INS waypoint. The mistake is not detected until well into the flight. Rather than fly an intercept to get back on the original track (which may waste fuel, at a premium), the captain decides to fly by dead reckoning, setting the autopilot to "heading select" mode, then flying the flight plan headings in a parallel course (but farther north) until he encounters an in-land radio navigational aid and can conveniently reset the flight plan. This behavior would suggest a lack of comfort with the INS (or, perhaps, a triple INS failure), or an unwillingness to deviate significantly from the paper flight plan and all of its nice pre-calculated values. He happens to intrude Soviet airspace at about the same time that a USAF E3A is expected, and gets shot down. The visual profile of the 747 is almost identical with that of the 707 (this is not as improbable as it sounds). Now, how does all of this relate to RISKS? We have the obvious entry error, which most of the theories surrounding the incident seem to accept. So, we say: develop a better entry mechanism. Easier said than done. More importantly, we can ask: why didn't the aircrew determine that they were off course? They certainly had enough information to determine the fact, assuming that they were following accepted crew practices. And, if they detected that they were off course about the time they started flying the parallel-but-too- far-north course, why didn't they get back on course? We might blame the highly automated environment. The operator error starts the ball rolling. The tedious, fatiguing long-distance Pacific run. The overreliance of the aircrew on the technology. The apparent incapacity to place importance on the fact that they were off course: in the insulated airliner environment, they might have concluded that a ten-minute deviation from course wasn't terribly significant, as long as they flew the phantom course defined by the flight plan. This "insulated" mentality is quite possibly a result of degraded flying skills from flying the automated environment too long. Over the years, I have seen behavior and read accounts of incidents that could account for or support all of the above. The design of cockpits is an exceedingly important issue, both from short-term performance considerations and those of long-term behavior modification. As numerous incidents have shown, automated cockpits remove the pilots from the control loop. When that happens, and, after 10,000 trouble-free flying hours, an insidious error occurs, the crew might not be able to compensate. This problem is due to shortly become MUCH more serious, with the advent of the two-man MD-11 and 747-400, both of which have unprecedented ranges. A number of foreign airlines like the airplanes, but not the automation and flight crew configuration, as evidenced by significant objections from KLM, Singapore, and a variety of Japanese carriers. Robert Dorsett, University of TX at Austin Internet: mentat@walt.cc.utexas.edu UUCP:{ihnp4, allegra,decvax}!ut-emx!walt.cc.utexas.edu!mentat ------------------------------ Date: 5 May 88 17:49:45 GMT From: Anand Iyengar <22116@pyr1.acs.udel.edu> Subject: Micros & Airlines - A New Angle Although I know a lot has been said about portables and airplanes, I couldn't resist this new aspect from the Sunday, May 1st, "Philadelphia Inquirer". ** Section R (Travel), page 7 ** "Emergencies are routine for airport medical team" First came the loud tone on the walkie-talkie, then came the call, "Code yellow, code yellow." ... The emergency code had come this time from a Boeing 747 on its way in from Boston. A heavy computer keyboard had popped a latch on an overhead compartment and fallen out, striking a 35-year-old business executive on the head. ... The man was dazed, had difficulty talking, and complained of weakness on one side. A concussion seemed almost certain. They took his vital signs, placed a collar on his neck, maneuvered him carefully onto a special chair, and took him to the jetway where they started an IV and administered oxygen. A fire rescue team arrived, got the patient onto a backboard, and headed for Methodist hospital... Just one more danger of these new-fangled machines. ------------------------------ Date: Wed, 04 May 88 18:18:06 -0700 From: "David A. Honig" Subject: Ollie North Helps PROFS sales Source: Computerworld "Inside Lines" May 2 1988 According to Paul Hessinger, Chief Technical Officer at Computer Task Group in Buffalo NY, "IBM received the largest number of orders ever for its Professional Office System, or Profs in the 14 days after Col. North's testimony! Prof's backup files had foiled North's shredding of certain communications during the "Iran-Contra Affair". ------------------------------ End of RISKS-FORUM Digest ************************