23-Jan-87 14:46:56-PST,16462;000000000000 Mail-From: NEUMANN created at 23-Jan-87 14:43:54 Date: Fri 23 Jan 87 14:43:54-PST From: RISKS FORUM (Peter G. Neumann -- Coordinator) Subject: RISKS DIGEST 4.42 Sender: NEUMANN@CSL.SRI.COM To: RISKS-LIST@CSL.SRI.COM RISKS-LIST: RISKS-FORUM Digest Friday, 23 January 1987 Volume 4 : Issue 42 FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: A scary tale--Sperry avionics module testing bites the dust? (Nancy Leveson) Computer gotcha (Dave Emery) Re: Another Bank Card Horror Story (Robert Frankston) Stock Market behavior (Howard Israel, Gary Kremen) Engineering models applied to systems (Alan Wexelblat) Re: British EFT note (Alan Wexelblat) Train Wreck Inquiry (Risks 2.9) (Matthew Kruk) Cost-benefit analyses and automobile recalls (John Chambers) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, nonrepetitious. Diversity is welcome. (Contributions to RISKS@CSL.SRI.COM, Requests to RISKS-Request@CSL.SRI.COM) (Back issues Vol i Issue j available in CSL.SRI.COM:RISKS-i.j. MAXj: Summary Contents Vol 1: RISKS-1.46; Vol 2: RISKS-2.57; Vol 3: RISKS-3.92.) ---------------------------------------------------------------------- Date: 21 Jan 87 12:53:51 PST (Wed) From: Nancy Leveson To: risks@csl.sri.com Subject: A scary tale--Sperry avionics module testing bites the dust? I just spoke to a man at the FAA who is involved with aircraft certification. He told me that Sperry Avionics, who are building the computerized automatic pilot among other things for a future new aircraft, is trying to convince them to eliminate module testing for the software. According to this man, Sperry argues that programmers find module testing too boring and won't stay around to do it. Instead of module testing, Sperry wants to use n-version programming and perform only functional system test. As long as the results from the two channels match, they will assume they are correct. I am not concerned that they are using n-version programming, but that they are arguing that the use of it justifies eliminating something that is considered reasonable software engineering practice. The FAA has agreed to allow them to try this. According to my FAA source, the FAA is not thoroughly comfortable with this, but the autopilot is only flight-crucial on this aircraft during about 45 seconds of the landing. Also, their tests have found that pilots can successfully recover from an autopilot failure during this period (by performing a go-around) about 80% of the time. I am going to talk further about this with some people at the FAA who are involved with certification. If anyone else shares my concern (or would like to allay my fears), I would appreciate hearing your opinions and arguments. I will convey them to the FAA unless you state that they should remain confidential. Nancy Leveson (nancy@ics.uci.edu) (P.S. Anybody want to join me in writing Congress about saving Amtrak?) ------------------------------ Date: Tue, 20 Jan 87 15:12:04 est From: emery@mitre-bedford.ARPA (Dave Emery) To: neumann@sri-csl.ARPA Subject: Computer gotcha ReSent-To: risks@CSL.SRI.COM Here's a computer gotcha for you... Like many other people, I was trying to close on a new house before the end of the year, for tax reasons. We had our down payment wired from our bank in New Jersey to our bank in New Hampshire, supposedly a fail-safe transaction. Unfortunately, the Bank of New England, which was (one of) the middleman in the wire transfer failed. Apparently, their system was overloaded, and crashed. My mother is a teller in a bank in Pittsburgh. She says that, at least at her bank, system crashes are a way of life. Fortunately, she says, they rarely lose any money. Dave Emery, MITRE Corp, Bedford MA P.S. Bank of New England recovered later that day, and we got our money after we signed the papers. The legal transaction was recorded the following day. ------------------------------ Date: Tue, 20 Jan 87 23:42 EST From: Frankston@MIT-MULTICS.ARPA Subject: Re: Another Bank Card Horror Story To: risks@CSL.SRI.COM The issue of ATM accountability reminds me of a problem I am having untangling my Mastercard transactions here. In general, the reports generated on the statements fail to provide the minimal information necessary for untangling messes. Information like which card was actually used is entirely missing. Only American Express seems to understand that each instance of a card should be tagged. This is especially annoying when the bank doesn't seem to mind that a stolen card is used for 8 months to buy tickets on the Eastern Shuttle. Credit transactions don't give a hint as to what transactions they are being counted against. While some of this just reflects ineptness and neglect in the bank's DP department, it also is indicative of what is going to become a real issue as we attempt to connect our personal computers with existing services. (or even banks connect to each other). Electronic banking services in general do an inadequate job of export/import of data. Such concepts as unique ID's for tagging and tracking transactions don't really exist. In the previous mastercard card example, the transaction id is just some characters associated with the transactions and are likely to not be unique. It seems as if my home processing of this information is more sophisticated than the bank's! It reminds me of my attempt to setup an equipment tagging system. I decided to order two sets of tags -- red for permanent stickers and black for removeables so that we can tag loaner equipment. The office manager followed through on this but both sets were numbered from 1 to 1000. It was difficult to explain why this was a problem since it was obvious which was red and which was black. The problems are manifestations of the issue of fundamental information processing literacy. While some of us working with computers have learned techniques to deal with aspects of this, the knowledge is not well distributed through society, nor even the DP profession. But the use of computers is becoming pervasive. This conflict is at the heart of a large class of computer-based risks. In the short term, the best we can do is point out the issues. Pointing out solutions is harder -- especially when they are obvious to us. The real question is how we can convey this understanding to the society at large. Are there any references that exist to try to explain the concepts of dealing with complicated systems and their interactions? Maybe even gather a list of such obvious things as checksums (and limitations on simple checksums), unique ids (and the low cost of using a lot of integers), redundancy (and its low cost/benefit) etc. On the other hand, maybe these difficulties are really blessings since an efficient EFT system, for example, might be a serious threat to privacy so that these annoyances and even risks are worth it till we understand how to deal with the system once it works smoothly. ------------------------------ Date: Tue, 20 Jan 87 16:17 EST From: Howard Israel Subject: RISKS 4.41, Stock Market behavior To: risks@CSL.SRI.COM >For the previous two witching hours, and for the forseeable future, market >makers are now required to publish their required major stock trades several >hours in advance on these Fridays. Minor correction. According to the Washington Post business section that described this new SEC strategy of disclosure, it is not "required", but recommended. All major brokerages complied except one (I believe it was Drexel Burnham Lambert Inc.), which caused a minor fervor as traders acted on "incomplete information", thus giving Drexel a slight advantage. Drexel was criticized for not disclosing their intended trades but countered that it did not violate any SEC "rules" and thus acted properly. The intended affect of the disclosure is to give traders advance notice, in effect, reducing the "shock" factor as well as allowing the "market makers" to adjust their inventories of stocks to prepare for the expected orders. Note: that a trade can be put in by a trader to be executed "at the market closing price" for a given stock. Regardless of what the price is, the trade will be executed. The deluge of orders on the "triple witching hour" at the "market closing price" often caused the ticker to be delayed up to a half hour at the closing. ------------------------------ To: sdcrdcf!hplabs!ucbvax!CSL.SRI.COM!RISKS Cc: risks@csl.sri.com, wanginst!infinet!rhorn@harvard.harvard.edu Subject: Stock Market behavior Date: Wed, 21 Jan 87 13:48:33 -0800 From: kremen@aerospace.ARPA >The problem of the ``triple witching hour'' is that during a few hours on >the third friday of each third month (typically from 3-4PM) there is an >immense burst of market activity as major participants rearrange their >computer selected portfolios. (This particular time is triggered by the >expiration time of a key financial component in these ``computer based'' >trades.) Not really true, most of the problem occurs in the last 10 minutes of trading when the "unwinding" of stock index futures, options on those futures, and the underlying equities occur. Usually the brunt of the unwinding occurs in Chicago, where the futures are traded. We only see a portion of this when one looks at the volume on the New York Stock Exchange. Also, not all unwinding occurs on "expiration day". If conditions are favorable, stock positions can be unwound earlier. >Before these trading programs, hearing that someone needs to sell 100,000 >shares of IBM quick, like in the next 5 minutes, meant that there was a >major problem at IBM. Many people still react in panic when they hear >such news. NO ONE panics. Since 1982, when stock market indexes (such as the Major Market Index or the S&P 100) started to be traded, the "triple witching hours" have occurred. Only within the past two years have the underlying markets been liquid enought to make it really worthwhile. Anyway institutions frequently sell (or buy) 100,000 shares of IBM for normal trading purposes. [...] For more information see the December 29, 1986 issue of Insight magazine. ------------------------------ Date: Tue, 20 Jan 87 10:49:53 CST From: Alan Wexelblat To: risks@csl.sri.com Subject: Engineering models applied to systems In Burnham's _The Rise of the Computer State_, MIT Professor Jeffrey A. Meldman is quoted as follows: "In engineering, there is a principle which holds that it is frequently best to have a loosely-coupled system. The problem with tightly coupled systems is that should a bad vibration start at one end of the machine, it will readiate and may cause difficulties in all parts of the system. Loose coupling is frequently essential to keep a large structure from falling down. I think this principle of mechanical engineering may be applicable to the way we use computers in the United States." The context of the quote is a chapter on the aggregations of power that can accrue in large, centralized computer systems and the risks (and temptations) of abuse of this power. RISKS readers have previously dismissed other engineering models as inapplicable to software systems. Comments on this one? Alan Wexelblat ARPA: WEX@MCC.ARPA or WEX@MCC.COM UUCP: {seismo, harvard, gatech, pyramid, &c.}!ut-sally!im4u!milano!wex ------------------------------ Date: Tue, 20 Jan 87 10:54:42 CST From: Alan Wexelblat To: RISKS@CSL.SRI.COM Subject: Re: British EFT note It is worth reminding RISKS readers that a British "billion" is a million millions (1,000,000,000,000) rather than the American thousand millions (1,000,000,000). --Alan Wexelblat [This was also noted by Howard Israel. By the way, I observe that the BBC radio broadcasts on PBS now routinely use "thousand million" and "million million"... PGN] ------------------------------ Date: Fri, 23 Jan 87 08:46:12 PST From: Matthew_Kruk%UBC.MAILNET@MIT-MULTICS.ARPA To: RISKS-Request@SRI-CSL.ARPA Subject: Train Wreck Inquiry (Risks 2.9) Just caught bits and pieces on the morning radio news about this item mentioned in Risks 2.9: An inquiry into the collision between a VIA passenger train and a Canadian National freight train near Hinton, Alberta last year, has put the blame on human error. The freight crew were said to have ignored various safety procedures. Also, Canadian National was accused of ignoring too many minor safety infractions and for letting crews work without sufficient rest periods between shifts. Computer error was not mentioned as a contributing factor. ------------------------------ From: harvard!cdx39!jc@seismo.CSS.GOV Date: 23 Jan 87 19:06:07 GMT To: mod-risks@seismo.CSS.GOV Subject: Cost-benefit analyses and automobile recalls (RISKS DIGEST 4.39) From: jc@cdx39.UUCP (John Chambers) > Moreover, even if the [cost-benefit] analyses were performed > correctly, the results could be socially unacceptable. [...] In the > case of automobile recalls, where the sample size is much larger, the > manufacturers may already be trading off the cost of a recall against > the expected cost of resulting lawsuits, although I hope not. Sure they are. Have you ever heard of "liability insurance"? There is also the general observation that, the way most forms of cost-benefit analysis work, ignoring (i.e., failing to assign an explicit value to) some factor is mathematically equivalent to assigning it a cost of zero. In other words, a cost-benefit analysis can't generally distiguish an unknown cost from a zero cost. Similarly for benefits. > The "value of a human life" is not a constant. The life of a volunteer or > professional, expended in the line of duty, has always been considered less > costly than the life of innocents. Huh? Most military organizations that I've heard of consider the cost of training a soldier to be significant; the value of innocents (i.e., civilians) is generally ignored, and thus considered to be zero. This was painfully obvious during the Vietnamese war, for instance. > As for the dangers of incorrectly estimating risks, I think that the > real danger is in not estimating risks. If you listen to public discussions of risky situations, it soon becomes quite clear that few people are able to distinguish "We know of no risks" from "We know there are no risks". ... > One can only imagine the reaction of the program authors when they > discovered what one last small change to the program's scoring function > was necessary to make it match the panel's results. It raises > interesting questions of whistle-blowing. Even if the programmers looked at the regression function, it's not clear that they would have even seen a racial component. For a nice example of how things can go wrong, consider that you can do a rather good job of sex discrimination while totally ignoring sex; you just use the person's height instead. There have been published studies that imply that this is widespread practice; it's been known for some years that [in the USA] a person's height is a better predictor of their income than their sex, and if height is included, knowing their sex adds no further information. It's likely that in the UK there are several attributes that are strongly correlated with race, so you need not necessarily use race at all. John M Chambers Phone: 617/364-2000x7304 Email: ...{adelie,bu-cs,harvax,inmet,mcsbos,mit-eddie,mot[bos]}!cdx39!{jc,news,root,usenet,uucp} Codex Corporation; Mailstop C1-30; 20 Cabot Blvd; Mansfield MA 02048-1193 ------------------------------ End of RISKS-FORUM Digest ************************ -------