Subject: RISKS DIGEST 12.52 REPLY-TO: risks@csl.sri.com RISKS-LIST: RISKS-FORUM Digest Monday 21 October 1991 Volume 12 : Issue 52 FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: The Future is Here (Amos Shapir) The_RISKS_of_Geraldo (Andy Hawks) [totally accidental juxtaposition!] Re: Police raid wrong house -- for second time (Amos Shapir) Re: TRW (Bob Colwell, Anthony DeBoer, Steve Hollasch) Re: buggy software (Mark R Cornwell, James B. Shearer, Byron Rakitzis, Richard Hanlon, Stephen G. Smith, Bob Wilson, David Parnas, David Chase) Licensing Software Engineers (Christopher E Fulmer) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, and nonrepetitious. Diversity is welcome. CONTRIBUTIONS to RISKS@CSL.SRI.COM, with relevant, substantive "Subject:" line. Others may be ignored! Contributions will not be ACKed. The load is too great. REQUESTS please to RISKS-Request@CSL.SRI.COM. For vol i issue j, type "FTP CRVAX.SRI.COMlogin anonymousAnyNonNullPW CD RISKS:GET RISKS-i.j" (where i=1 to 12, j always TWO digits). Vol i summaries in j=00; "dir risks-*.*" gives directory; "bye" logs out. The COLON in "CD RISKS:" is essential. "CRVAX.SRI.COM" = "128.18.10.1". =CarriageReturn; FTPs may differ; UNIX prompts for username, password. ALL CONTRIBUTIONS CONSIDERED AS PERSONAL COMMENTS; USUAL DISCLAIMERS APPLY. Relevant contributions may appear in the RISKS section of regular issues of ACM SIGSOFT's SOFTWARE ENGINEERING NOTES, unless you state otherwise. ---------------------------------------------------------------------- Date: Thu, 17 Oct 91 17:45:36 +0200 From: amos shapir Subject: The Future is Here The Israeli Broadcasting Authority (IBA) is an independently budgeted government agency, financed by a special tax (sometimes called, for historical reasons, "TV license fee"). Since it does its own collection, it's sometimes even more zealous than the IRS. The following is a true incident that happened to a friend of mine: She received a notice from the IBA to pay back due taxes. The strange thing about it was that it was sent to an address she moved into just ten days before, and was sure nobody but the landlord knew about. While she was in the IBA's office to settle the matter, she'd found out that due to recent unification of government databases, the following information about her was retrieved at a touch of a clerk's terminal key: - Her new address (probably from the city's municipality); - The type of each TV she'd owned since 1984 (dealers and importers are required to report that to the IBA); - The dates in which she'd left and entered the country during that time (probably from the border police passport control records). All this was just information necessary for the specific case at hand; that Big Computer probably knows a lot more about us. In short, the future is here, and it looks more Orwellian than Orwell could have ever imagined. Amos Shapir, The Hebrew Univ. of Jerusalem, Dept. of Comp.Science. +972 2 585706 ------------------------------ Date: Wed, 16 Oct 91 18:41:23 MDT From: ahawks@isis.cs.du.edu (Andy Hawks) Subject: The_RISKS_of_Geraldo I'm sure many of you saw or have heard/read about Geraldo Rivera's Now It Can Be Told Program which featured a show on hackers a couple of weeks or so ago. Well, by airing this program, it appears that Geraldo (or actually the producers/editors of the show) have put at least one military computer at risk. One segment of the program featured a "home video" of Dutch teenagers hacking. This home video would occasionally focus in on the computer screen as the hackers hacked. As reporter Krista Bradford describes what is going on, the screen shows: > | quit | 221 Goodbye. | rugrcx> | telnet tracer.army.mil | Trying 192.33.5.135.... | Connected to tracer.army.mil | Escape character is '^]'. | | Xenix K3-4 (tracer.army.mil) | | login: | dquayle | Password:_ > Then we learn that previously, the hackers have gained superuser privileges to the system. As Krista Bradford is describing the superuser access, we see the computer screen again and the hackers are attempting to login to the same site with the 'sync' login (so, this is apparently how they gained superuser access). Later in the show (about 1 minute or so after the hackers have gained superuser privileges) Emmanuel Goldstein (2600) states that the hackers proceeded to create a new account. The account they create is 'dquayle' (Dan Quayle) and has superuser privileges. Then, the screen focuses in on the new record in /etc/passwd for 'dquayle', and Mr. Goldstein tells us that the new account has no password (the screen focuses in on: "dquayle::") Thus, anyone who has telnet access could've repeated this same process, logging in to this tracer.army.mil site with the username 'dquayle' (and no password) and would have gained superuser access. It is obvious that in this situation, whoever allowed the show to be aired in its final form had no knowledge of the Internet, otherwise this definite "how to hack" security breach would have been omitted. Thanks Geraldo, for showing all of us how to hack into military computers. (Note: I avoided sending this in for submission earlier to prevent any other hackers from repeating the same experiment. Hopefully, tracer.army.mil has now had enough time to plug up the obvious hole.) ------------------------------ Date: Thu, 17 Oct 91 17:39:41 +0200 From: amos shapir Subject: Re: Police raid wrong house -- for second time [Quoted from referenced article by dbenson@yoda.eecs.wsu.edu (David B. Benson)] > The officers didn't leave until Dean Krussel showed them >Callahan's letter. "This thing just won't go away," he said This seems to be a Law of Nature in computer systems: Nothing ever goes away. Many databases keep each datum as an initial entry enhanced by a set of update records (I suspect some even run through the whole update process every time they're rebooted). Every now and then a system crashes, someone loads a wrong backup tape, etc., and voila! your magazine is being sent to an address you have left 12 years ago, or your house is being raided by the police... :-( Amos Shapir The Hebrew Univ. of Jerusalem, Dept. of Comp. Science. 972 2 585706 ------------------------------ Date: Thu, 17 Oct 91 09:08:57 -0700 From: colwell@ichips.intel.com Subject: TRW in chaos? No darn wonder the general public thinks the techies are clueless and screwing up the world. In OMNI's August 1991 issue, Kenneth Hey says (pg. 84): "While human misuse of technology has reached bothersome levels [personally, biased and twisted thinking like this article bother me a lot more], technology can also assume a worrisome life of its own. TRW computer designers expressed surprise when a large network of computers they created began exhibiting 'strange, unpredictable' behavior. During these periods, the system could not perform specific tasks as requested. TRW suspected 'chaos', an uncontrollable but natural mathematical phenomenon, which mysteriously attacks complex computer systems. Scientists at the Xerox Palo Alto Research Center conducted a series of experiments and discovered that, indeed, large aggregates of connected computers can exhibit unpredictably wild oscillations and unstable behavior, generating unwanted actions in the system. The reality of computer instability -- that is, the real potential for chaotic behavior -- has raised professional concerns about the appropriate level of computer dependence for military, corporate, and informational systems." Maybe the TRW system was just busy writing this article for OMNI...nah, it would have turned out a lot better. Anybody from TRW care to shed any light on what this person was talking about? 503-696-4550 Bob Colwell, Intel Corp, 5200 NE Elam Young Parkway, Hillsboro, OR 97124 ------------------------------ Date: Thu, 17 Oct 1991 09:28:57 -0400 From: adeboer@gjetor.geac.com (Anthony DeBoer) Subject: Re: TRW misreports local taxes (Seecof, RISKS-12.50) Why is it that inaccurately negative credit reporting [doesn't | shouldn't] constitute libel under the law? It would seem that if someone told a third party all kinds of horrible things about me that weren't true, at a certain point a line would be crossed and they'd be liable for damages. What is TRW's defense for this? ------------------------------ Date: Thu, 17 Oct 91 11:00:34 PDT From: hollasch@kpc.com (Steve Hollasch) Subject: TRW Credit Reports In RISKS 12.51, Rob Spray gave the procedure necessary to obtain a free credit report from TRW (TRW recently ``decided'' to send free credit reports to people who wish to see their file, as maintained by TRW). Included in the description of the procedure to obtain this report is the following list of information that TRW needs before it will send it to you: - Full name - Spouse's first name - Addresses with zip codes for last five years - SSN - DOB It seems like catch-22 that people who are concerned about privacy are required to send this information in order to check their records. On thinking about this list, though, it also becomes apparent that TRW must request some private information to verify that the requester is authentic. Is it reasonable to assume that TRW already has all this information? If this is true, then my recent attempts to keep my SSN private seem rather futile. Why bother keeping something private when it is available in a public database (albeit for sale)? What information is available to those who subscribe to this service? Steve Hollasch, Kubota Pacific Computer, Inc., Santa Clara, California ------------------------------ Date: Thu, 17 Oct 91 13:32:26 -0400 From: Mark R Cornwell -- Mind Tools Corp Subject: Re: buggy software (RISKS-12.50) and Steinmetz In recent exchanges between David Parnas (12.45,50) and James Shearer (12.49) the issue of who should assume the risk of software bugs has arisen. Let me say at the outset that I dislike the term "bug". I will use the term "error". It is not necessary to make a blanket judgement about who should assume the risk of programming errors. A software license can be structured so that either the client or the vendor assumes this risk. The decision is best left to them. The public then has recourse against the party who has agreed to assume the risk. That said, I think that it would be better for the software profession if standard practice were that the vendor assume the risk of his programming errors. I know of no better way to create an incentive for vendors to provide quality software. If software vendors chose to be accountable for programming errors in their products, they might be willing to try more "crackpot" ideas in place of a process that few believe is working well. Last night I was at a gathering of local entrepeneurs speaking with an independent software developer. He is president of his own corporation. He writes programs. He described his work with XWindows and Motif on the latest workstations in manufacturing. He asked me what kind of mathematics I studied and I started to tell him about correctness proofs of programs. When I told him that a program could be viewed as a function from states to states he was fascinated. He said he had never though of programs like that before, but the idea appealed to him. I felt like Steinmetz must have felt talking to an early engineer who built electric power systems. "You can think of the current plotted against time as a sine wave". "Very interesting, I'd never thought to look at it that way before." --Mark Cornwell [OK. Time to blow the whistle on this subject, after the following messages... PGN] ------------------------------ Date: Wed, 16 Oct 91 20:22:22 EDT From: jbs@watson.ibm.com Subject: Re: buggy software (RISKS-12.49,12.50) In reply to my post suggesting buggy software is not a major threat to the republic David Parnas writes in part: "We could use the "mature adult" excuse to get rid of all of these regulations, but we would all be worse off for doing so. Your apartment could be destroyed because one of your "mature adult" neighbours bought an appliance that was not properly designed. Your child could be injured because one of your "mature adult" neighbours bought a car with defective brakes. Further, every time you bought one of those products you would have to determine its safety for yourself, whether you knew enough to do so or not." And what, pray tell, terrible disaster will befall me when my neighbor buys buggy software for his pc? Also if Mr. Parnas believes that the regulation of autos and electrical appliances means there is no need for a buyer to consider their safety he is sadly mistaken. Tens of thousands of people are killed using autos every year. This is somewhat more than are killed using computer software. David Parnas also writes: "Those who object to the suggestion that software products should be subject to safety requirements and that software manufacturers should be held responsible for the results of any negligence seem to believe that we are asking for special treatment of software. Au contraire! We are asking that software be treated like other products, produced by registered or licenced engineers, and that software manufacturers be treated like other manufacturers. Now, because of the supposedly non-physical nature of software, programmer's products seem to have special exemption. If cars were as buggy as the software on the market today, the automobile manufacturers would have long ago been sued into bankruptcy." Mr. Parnas appears to have things backward. So far as I know software is currently treated like any other product in the uniform commercial code and other general laws regulating commerce. Mr. Parnas is asking that software be subject to additional special regulation like that imposed on certain hazardous products such as cars despite the fact that in most cases software defects pose no equivalent direct danger. James B. Shearer ------------------------------ Date: Wed, 16 Oct 1991 23:47:55 -0500 From: Byron Rakitzis Subject: Re: buggy software (Parnas, RISKS-12.50) Were it not for the hauteur in this posting, I would have let this go by. But let me just state: there are a number of us who believe that the product liability laws have gone way past any reasonable point. What used to be governed by contract law is now covered by the law of torts in the US, with the pervasive motif being "it's the rich guy's fault". Please keep the hysterics to a minimum and try to assess what product liability laws have to offer: there is some added factor of safety at a huge cost. Witness the skyrocketing costs of medical insurance. Witness the fact that pharmaceutical companies are most reluctant to release new products, and for example have all but halted research on contraceptives. Another snag is that liability is not determined by experts, it is determined by a JURY (at least in these United States). How is a jury going to make a reasonable decision on the alleged defectiveness of FooNix? And if some such laws come to pass, what software developers in their right minds will market products like FooNix? Will we all not be somewhat the poorer for this? No doubt the intentions of such laws are noble: to protect the ignorant (your implication, not mine) public from being duped by unscrupulous business. However, such protection comes at a huge cost in liberty, money and time, and this cost should not be so callously dismissed. ------------------------------ Date: Thu, 17 Oct 91 12:45:56 -0700 From: d3e198@bsip54.pnl.gov Subject: self regulations for buggy software instead of gov. regs Although I believe that both Mr. Kempe (RISKS-12.51) and Mr. Parnas (RISKS-12.50, 51) have good points, I believe Mr. Thomas (RISKS-12.50) points out a greater risk that our profession may be painting itself into a corner. After the recent failures of computer systems (AT&T's phone outage causing airline travelers to be delayed, the stock market problem which I can't remember when it happened, plane crashes, etc.) it would only take a headline like Computer Glitch Causes Nuclear Power Plant to Meltdown, XXX People Evacuated, State Declared Off-Limits for Next 10,000 Years, or Computer Glitch Causes Missile Launch, XXX People Killed before the world's public would demand a reckoning (witch-burning comes to mind). Instead, perhaps our profession should try to become more _self_ regulating. If we clean up our own act, _before_ the government can step in, then it will be possible to set the regulations ourselves, instead of the *all knowing* committees of governmental bureaucracy. Just as doctors are mostly self-regulating, so should the computer science field be. For example, as Mr. Massey suggests, the term software is used as a conglomerate of a large, diverse genre of products. But I feel that the computer science field itself should break it apart and do the categorizing. If we do not, someone else surely will, and will probably not do it in a way that most of the rest of us like. Computers now control major parts of our lives e.g. airline safety, automobiles, medical systems, nuclear power plants, etc. (Whether this control is good or bad, or whether the manner in which they do the controlling and interacting with humans is good or bad is another topic.) They can easily do massive damage through negligence on the part of the software, design, or through hardware failure. This risk of disrupting (even endangering) our lives makes a great need for regulation as Mr. Parnas suggests. The public has a reasonable expectation of safety and reliability, and the only way to meet this expectation is through some sort of standardization. But as Mr. Kempe points out, pointing a gun at a programmer's head does NOT produce necessarily good code. Forcing people to do things usually ends in failure. The government is definitely not the body to do any regulating. I further agree that software engineering is an art. The U.S. Court systems have been treating software as books in the regards to the copyright laws. Many license agreements liken to their software to books. [Therefore you must treat this software just like a _book_ with the following single exception. ... make archival copies of the software for the sole purpose of backing-up your software and protecting your investment from loss.] To avoid the problem of *Big Brother* watching the programmers, and to avoid taking the art out of the software engineering, a self-adopted code of *ethics* can satisfy both the regulations _and_ the art. If one feels pride in the following of a such a code, then there is a greater chance that the person will continue to follow this code willingly. It becomes a matter of personal integrity. This system is not fool proof, e.g. how many doctors are sued in mal-practices suits each year, but nobody quits going to the doctor because he then believes that all doctors are quacks. Self-adopted codes would need validation of course. This could be provided with a minimal intrusion by the government, making it a law to provide free bug fixes (there's ALWAYS at least one more bug). From there on, the economic laws of supply and demand based on quality should enforce adherence to the regulations. For example, if company X develops a reputation as having very reliable software the first time around, then people will tend to buy X's software, just as is done in the market for other products. Richard Hanlon ------------------------------ Date: Thu, 17 Oct 91 16:38:40 -0400 From: sgs@grebyn.com (Stephen G. Smith) Subject: Re: buggy software In RISKS DIGEST 12.51, skewes@CAD.MCC.COM (Ernesto Pacas-Skewes) writes: >I'm just saying that registration and license like so many other >things aren't always what they seem. Ain't it the truth! In particular, I see registration, licensing, etc, as being an attempt by the companies that write software to limit their liability. By having a "licensed professional" sign off on their software, they hope (IMHO!) to be able to say "This software was produced according to standard industry practice" when they produce the next Therac-25. Unfortunately, there is no "standard industry practice" that will provide an assurance of good software, other than the blanket "good engineering practice". The main causes that I've seen for bad software are management issues, rather than technical issues. In particular: 1. Software always seems to be produced under *extremely* high stress. Only medicine and (possibly) law require high performance under higher stress conditions. This is usually caused by assorted forms of bad scheduling. 2. Many software managers fancy themselves as techies, despite the fact that they may not have ever written a line of production code. The urge to micromanage seems to be irresistable. 3. Specifications (IMHO, the most important part of the project) are often confusing, conflicting, or incomplete. "Formal verification" against a bad (English!) specification is, at best, a waste of time. So how can we improve management? I dunno. Most of the outfits that I've worked for were strictly "top-down" -- directives come down from the top and status information goes up. Statements from the troops like "No way can this get done on time" tend to get lost. Steve Smith, Agincourt Computing sgs@grebyn.com (301) 681 7395 ------------------------------ Date: Thu, 17 Oct 91 16:21:11 CDT From: Bob Wilson Subject: Re: Bart Massey on "Buggy Software" Bart Massey believes you could make a case for licensing software production for medical software, etc., for which he sees a "threat of bodily harm", but he goes on that "it is clear that" no safety supervision is needed for computer games or word processors because they can make no such threat. (Of course "it is clear that" is a dangerous phrase to dangle in front of a mathematician...) He may be right about a threat of BODILY HARM, but there are other very real threats. RISKS has had several examples of potential harms from word processors, to take one of his examples. Many of us surely write memos or letters we need to go back and tone down: The original, if published, might be dangerous at least to our economic health. We have seen here in the last several years examples of commercial word processors which would retain in the disk version of your document what you had deleted from the printed version, or for that matter what might have been in a disk block unrelated to the present document. Those contents are not hard to look at, if your employer sends somebody in to see what you have been saying about your boss or the company. The controversy over Prodigy and whether it was "stealing" copies of things from your system represents a RISK in a system frequently used for game playing. Regardless of whether Prodigy was doing it, the RISK is there for some other communications related software to exploit. I don't like the idea of licensing software engineers, but I think it is too simplistic to think the only dangerously RISKy software is that used for medical instruments, nuclear power systems, and vehicle controls. Those make nice examples because they are so far beyond question, but by the same token they also get more scrutiny. That scrutiny may not be enough, but it surely doesn't mean we can ignore the RISKS in more mundane applications. Bob Wilson, Math Dep't, Univ. of Wisconsin ------------------------------ Date: Fri, 18 Oct 91 17:42:39 EDT From: parnas@qusunt.eng.McMaster.CA (David Parnas) Subject: Re: buggy software (RISKS-12.51) Bart Massey claims there is no risk of harm from buggy wordprocessing software. If that software is used to produce the manual for a dangerous device,` and deletes important warnings, there is a risk. I agree with Mr. Massey's attempts to draw lines and think it is obvious that there are shades of grey in this area, but the analysis is more difficult than it looks. I am aware of all the weaknesses of licensing and registration raised by Ernesto Pacas-Skewes. I sometimes have to drive a car and observe what licensed drivers do with their properly registered (but inadequately equipped) cars. Nonetheless, when given a choice, I prefer drivers who did pass a driving examination to those who never were able to do so. Perhaps we should stop arguing about whether some regulation is ever needed and start to think hard about what should be regulated, how it should be regulated, and who should do the regulation. When we do, I think we will find useful ideas in other fields of engineering. Dave Parnas parnas@sscvax.cis.mcmaster.ca ------------------------------ Date: Thu, 17 Oct 91 19:58:31 EDT From: jbs@watson.ibm.com Subject: Re: buggy software (Parnas, RISKS-12.51) I suspect "variety of external pressures" means "competition", "if the market were better controlled" means "if the competition was put out of business" and "users were better informed about products" means "users stopped worrying about minor factors such as cost, per- formance, timeliness and function". James B. Shearer ------------------------------ Date: Thu, 17 Oct 91 17:20:54 PDT From: David.Chase@eng.sun.com (David Chase) Subject: re: buggy software (bart, RISKS-12.51) > ... There is probably some intermediate class of > software applications where a UL-like oversight body would be the > appropriate answer. This is a mere "difference of opinion", but I have certain expectations even of games and word processors. In particular, I expect that there are no bugs in the program which might result in the destruction of unrelated data stored on the same computer. (At this point, of course, the OS vendor and the games vendor engage in heated finger-pointing, and the customer is left grumbling.) This is a far cry from the reliability I expect from an airplane, or an EFT point-of-sale-terminal, but it is more than none at all, and I'd rather not verify it for myself. Furthermore, past experience indicates that "verification" is difficult and time-consuming, and not something that a customer is interested in doing. There is the second problem of "the use of this software". Both software and physical devices are both put to unintended uses. I can't think of any really juicy software examples just this instant, but once upon a time someone I know did use a bicycle cone wrench in place of a 70-amp slow-blow fuse (that had blown). David Chase, Sun ------------------------------ Date: Fri, 18 Oct 91 13:26:51 -0400 From: fulme-ce@lea.csc.ncsu.edu (Christopher E Fulmer) Subject: Licensing Software Engineers (still more) With all of the discussion going on about the RISKS associated with not having a mechanism for licensing software engineers, it seems to me that we are ignoring a few basic points..... 1. Licensing software engineers has the effect of reducing the number of people who are involved in software engineering. While this can be good (In the case of eliminating those who are incompetent), it also has the effect of putting the decision of "Who is competent" in the hands of some governing authority. And, that authority may have alterior motives for making their decisions. The authority would have to be made up of practicing software engineers. (Who else is qualified to judge?),who may very well desire to keep the field of software engineers down, by making the standards tougher, thus increasing their own marketability. (Dry cleaners have managed to do this by pushing through a "Certification of Dry Cleaners" law in some states.) 2. The market does tend to push out poorly-designed products. However, for some products, it may not be desirable to wait for the market to decide. After all, Audi's sales dropped after the problems with "Instant Acceleration" were found by real people, not before. 3. A mechanism which is typically used by Government and Commercial bodies is the idea of the "Contract model," wherein certain specifications are set up, and products which do not meet those specifications are not paid for. So, perhaps the solution to all this is not for the government to license softwre engineers, but to instead set up minimal specifications for safety critical applications. Or, perhaps, to provide for the independent licensing of the products, and not of the people who design them. In addition to solving the problems posed by #1 & #2 above, this also solves problems caused by bad software written by good people. Heck, certified engineers make mistakes all the time. What makes us think that programmers are different? So, in conclusion, It's my opinion that we're trying to solve the problem of poor software quality by looking at the people who wrote it, instead of looking at the produ of their work. Poor programmers, given enough time, can write perfectly good software, just as fantastic programmers in a rush can write terrible software. So, it's essential that we gauge the quality of the work, and not the quality of those who produced it. Chris Fulmer fulme-ce@lea.csc.ncsu.edu ------------------------------ End of RISKS-FORUM Digest 12.52 ************************