💾 Archived View for gemini.quux.org › h › Archives › mirrors › risks › illustrative.html captured on 2024-12-17 at 10:11:00.
⬅️ Previous capture (2024-08-25)
-=-=-=-=-=-=-
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<HTML>
<!-- HTML file produced from file: sigdish.tex --
-- using Hyperlatex v 2.3.1 (c) Otfried Cheong--
-- on Emacs 20.7.1, Mon Jan 15 15:43:53 2001 -->
<HEAD>
<TITLE>Untitled</TITLE>
</HEAD><BODY BGCOLOR="#ffffff">
<FONT SIZE=+1><BLOCKQUOTE><CENTER>
<B><FONT SIZE=+1>Illustrative Risks to the Public</FONT></B><BR><B><FONT SIZE=+1>in the Use of Computer Systems</FONT></B><BR><B><FONT SIZE=+1>and Related Technology</FONT></B> <BR>Peter G. Neumann, Computer Science Laboratory, <BR>SRI International, Menlo Park CA 94025-3493<BR></CENTER></BLOCKQUOTE>
<FONT SIZE=+0><P>Risks cases as of <B>15 January 2001,</B>
Copyright 2001, Peter G. Neumann, SRI
International EL243, Menlo Park CA 94025-3493 (e-mail Neumann@csl.sri.com;
<A HREF="http://www.CSL.sri.com/neumann/">http://www.CSL.sri.com/neumann/</A>;
telephone 1-650-859-2375; fax 1-650-859-2844): Editor, ACM SIGSOFT Software
Engineering Notes, 1976-93, Assoc.Ed., 1994-; Chairman, ACM Committee on
Computers and Public Policy (CCPP); Moderator of the Risks Forum
(comp.risks); cofounder with Lauren Weinstein of People For Internet
Responsibility (<A HREF="http://www.pfir.org">http://www.pfir.org</A>,
announced in RISKS-20.65). See also Lauren's Privacy Forum Digest
(<A HREF="http://www.vortex.com">http://www.vortex.com</A>),
partially sponsored by the ACM CCPP.
<H1><A NAME="1">Contents</A></H1>
<MENU>
<LI><A HREF="#1">Contents</A>
<LI><A HREF="#2">Descriptor Symbols</A>
<LI><A HREF="#3">Items Listed by Categories</A>
<MENU>
<LI><A HREF="#4">Recent Items (yet to be merged in)</A>
<LI><A HREF="#5">Space</A>
<LI><A HREF="#6">Defense</A>
<LI><A HREF="#7">Military Aviation</A>
<LI><A HREF="#8">Commercial Aviation</A>
<LI><A HREF="#9">Rail, Bus, and Other Public Transit</A>
<LI><A HREF="#10">Automobiles</A>
<LI><A HREF="#11">Motor-Vehicle and related Database Problems</A>
<LI><A HREF="#12">Electrical Power (nuclear and other) and Energy</A>
<LI><A HREF="#13">Medical, Health, and Safety Risks</A>
<LI><A HREF="#14">Other Environmental Risks</A>
<LI><A HREF="#15">Robots and Artificial Intelligence</A>
<LI><A HREF="#16">Other Control-System Problems</A>
<LI><A HREF="#17">Other Computer-Aided-Design Problems</A>
<LI><A HREF="#18">Accidental Financial Losses, Errors, System Outages</A>
<LI><A HREF="#19">Financial Frauds and Intentionally Caused Losses</A>
<LI><A HREF="#20">Stock-Market Phenomena</A>
<LI><A HREF="#21">Telephone Frauds</A>
<LI><A HREF="#22">Other Telephone and Communication Problems</A>
<LI><A HREF="#23">Election Problems</A>
<LI><A HREF="#24">Insurance Frauds</A>
<LI><A HREF="#25">Security Problems</A>
<LI><A HREF="#26">Cryptography</A>
<LI><A HREF="#27">April Foolery and Spoofs</A>
<LI><A HREF="#28">Privacy</A>
<LI><A HREF="#29">Spamming, Junkmail, and Related Annoyances:</A>
<LI><A HREF="#30">Other Unintentional Denials of Service:</A>
<LI><A HREF="#31">Law Enforcement Abuses, False Arrests, etc..</A>
<LI><A HREF="#32">Identity Theft, Mistakes, Related Problems</A>
<LI><A HREF="#33">Other Legal Implications</A>
<LI><A HREF="#34">Other Aggravation</A>
<LI><A HREF="#35">Calendar/Date/Clock Problems including Y2K</A>
<LI><A HREF="#36">The Game of Chess:</A>
<LI><A HREF="#37">Further Miscellaneous Hardware/Software Problems</A>
<LI><A HREF="#38">Other Computer System Development Difficulties</A>
<LI><A HREF="#39">Achieving Better System Development and Operation</A>
<LI><A HREF="#40">The Proper Role of Technology?</A>
</MENU>
<LI><A HREF="#41">Reference Materials</A>
<MENU>
<LI><A HREF="#42">Books</A>
<LI><A HREF="#43">Inside Risks</A>
</MENU>
</MENU>
<P>This list summarizes items that have appeared in the Internet Risks Forum
Digest (RISKS) - which I moderate (comp.risks newsgroup) - and/or
published ACM SIGSOFT Software Engineering Notes (SEN). In this collection
of mostly one-liner summaries, (R i j) denotes RISKS volume i issue j; (S
vol no:page) denotes an issue of SEN, where there has been one volume per
year, with vol 25 being the year 2000; page numbers are given primarily only
from 1993 on; (SAC vol no) indicates an item in the quarterly SIGSAC
Security and Control Review, where vol 16 is 1998, which was the final
volume. The SEN material prior to 1995 is summarized in my Computer-Related
Risks book (see below). Later material is gradually being brought on-line,
as noted below.
<P>Some incidents are well documented, while others need further study. A few
are of questionable authenticity, and are noted as such ("bogus???").
Please send me corrections and new cases, along with suitable references.
This list is updated at least quarterly and is browsable on-line
(<CODE>ftp://ftp.CSL.sri.com/neumann/illustrative.html</CODE> courtesy of Otfried
Cheong's Hyperlatex), also printable in a two-column 8-point format
(<A HREF="illustrative.pdf">illustrative.pdf</A> and
<A HREF="illustrative.ps">illustrative.ps</A>). [Incidentally, Hyperlatex
is wonderful Free Software:
(<A HREF="http://www.cs.ust.hk/~otfried/Hyperlatex/">http://www.cs.ust.hk/~otfried/Hyperlatex/</A>,
and after September 2000:<BR><A HREF="http://www.cs.uul.nl/~otfried/Hyperlatex">http://www.cs.uul.nl/~otfried/Hyperlatex</A>).]
<PRE>
SEN regular issues, by year, volume&number
..1976,vol 1: #1 = May; #2 = Oct
==================================
..year 1977 78 79 80 81 82 83 84 85
volume 2 3 4 5 6 7 8 9 10
---------------------------------
Jan #1 1 1 1 1 1 1 1 1
Apr #3 2 2 2 2 2 2 2 2
Jul #4 3 3 3 3 3 3 4 3
Oct #5 4 4 4 5 4 5 5 5
==================================
..year 1986 87 88 89 90 91 92 93 94
volume 11 12 13 14 15 16 17 18 19
---------------------------------
Jan #1 1 1 1 1 1 1 1 1
Apr #2 2 2 2 2 2 2 2 2
Jul #3 3 3 5 3 3 3 3 3
Oct #5 5 4 6 5 4 4 4 4
==================================
..1995,vol20: #1=Jan; 2=Apr; 3=Jul; 5=Dec
..1996,vol21: #1=Jan; 2=Mar; 4=Jul; 5=Sep
..1997,vol22: #1=Jan; 2=Mar; 4=Jul; 5=Sep
..1998,vol23: #1=Jan; 3=May; 4=Jul; 5=Sep
..1999,vol24: #1=Jan; 3=May; 4=Jul; 5=Dec
..2000,vol25: #1=Jan; 2=Mar; 3=May; 4=Jul
..2001,vol26: #1=Jan; 2=Mar;
</PRE>
<P>Read the Risks Forum as comp.risks if you can, or
send e-mail to risks-request@csl.sri.com for a subscription, single text
line "subscribe" (append desired address only if not your From: address), or
"info" for info. Send contributions to risks@CSL.sri.com. Archives are
available from <CODE>ftp://ftp.sri.com/risks</CODE> or by "ftp ftp.sri.com",
"login anonymous", "cd risks" (which gets the "dir" for the current volume,
and "cd i" then gets you into the subdirectory for any preceding volume i =
1 to 19). A mirror at Newcastle
<A HREF="http://catless.ncl.ac.uk/Risks/">http://catless.ncl.ac.uk/Risks/</A>
is maintained by Lindsay Marshall, and includes a nice search facility.
Specific issues can be read directly as
<CODE>http://catless.ncl.ac.uk/Risks/I.J.html</CODE> [where I=volume#, J=issue#].
An Australian mirror is at
<A HREF="http://the.wiretapped.net/security/textfiles/risks-digest/">http://the.wiretapped.net/security/textfiles/risks-digest/</A>.
"Inside Risks" distills some of the discussion into a
monthly inside-back-cover column in the <I>Communications of the ACM.</I>
The list columns to date is given at the end of this list.
<P>My book (Peter G. Neumann, <I>Computer-Related Risks,</I> Addison-Wesley
(ISBN 0-201-55805-X) and ACM Press (ACM Order 704943), 1995) summarizes many
of these cases and provides additional analysis. (A few errata for the
first three printings are on my Web page, noted above.) Most of the (S vol
no) items listed below for i <I><</I> 20 are discussed in the book; more recent
items generally include the relevant on-line (R i j) references. If you
cannot find the book in a bookstore, it is on amazon.com, or call A-W within
the U.S. at 1-800-822-6339 - or if you are outside of the U.S.,
1-617-944-3770 and ask for International Orders. The book is now also
available in Japanese (ISBN 4-89471-141-9). Instead of trying to produce a
second edition in the face of a massive influx of new RISKS cases, the
fourth printing of the book gives the URL for the Addison-Wesley Web site
(<A HREF="http:www.awl.com/cseng/titles/ISBN-0-201-55805-X/">http:www.awl.com/cseng/titles/ISBN-0-201-55805-X/</A>),
which includes the first chapter of the book and an extended preface. That
Web site and my own contain more material that would otherwise have gone
into the second edition. See
<A HREF="http://www.csl.sri.com/neumann/risks-new.html">http://www.csl.sri.com/neumann/risks-new.html</A> for new material
and
<A HREF="http://www.csl.sri.com/neumann/cal.html">http://www.csl.sri.com/neumann/cal.html</A>
for an excerpted summary of Y2K and related calendar-clock problems.
<P>Henry Petroski (among others) has noted that we rarely learn from our
successes, and must learn more from our failures. The collection of cases
cited here provides rich opportunities for reflection that could help us to
avoid similar problems in the future. Unfortunately, it also demonstrates
that the same types of mistakes tend to recur.
<P><I>SEN</I> and RISKS also consider approaches for developing better computer
systems, e.g., safer, more reliable, more secure. There are many approaches
to developing sound systems; none is guaranteed. Whereas the emphasis in
the following list is on problems rather than on would-be solutions, the
pervasive nature of the problems suggests that techniques for the effective
development and operation of computer-related systems are frequently
ignored. Worse yet, even ideal systems can result in serious risks, through
unanticipated technological problems or human foibles. We include here
primarily cases that have been publically reported, although we know of
various additional cases whose existence for one reason or another has not
seen the light of day. A few successes are also included, although the
failures seem to predominate. We are always interested in hearing more
about successes. Although I receive occasional complaints about the
preponderance of failures in RISKS, there appear to be very few real
successes. Perhaps not enough folks are heeding some of the advice that you
can gather from RISKS and that are distilled in <I>Computer-Related
Risks.</I>
<H1><A NAME="2">Descriptor Symbols</A></H1>
The following descriptor symbols characterize each entry.
<P>! = Loss of life/lives; * = Potentially life-critical or safety problem
<P>V = Overall system or subsystem sur<B>V</B>i<B>V</B>ability problems
(with respect
to diVerse adVersities, including attacks <I>and</I> malfunctions).
Startlingly many cases fit this category; many V-unflagged cases also
represent failures to continue performing properly, or delays, or other
cases of misuse that could have led to much more serious survivability
problems.
<P>$ = Loss of resources, primarily financial
<P>S = <B>S</B>ecurity/integrity/misuse problem;
P = <B>P</B>rivacy/rights abuse or concern
<P>H = Intentional <B>H</B>uman misuse (e.g., user-administrator-operator-penetrator)
<P>h = Accidental <B>H</B>uman misuse or other inadvertence
<P>I = <B>I</B>nsider; O = <B>O</B>utsider; A = Inadequate <B>A</B>uthentication or
<B>A</B>ccess controls
<P>d = System <B>D</B>evelopment problems
<P>e = Improper <B>E</B>volution/maintenance/upgrade. (H,h,i,f,d,e involve human foibles.)
<P>r = Problems with <B>R</B>equirements for system or
operation (including the overall system concept)
<P>f = <B>F</B>laws (or <B>F</B>eatures in design, or hardware/software
implementation
<P>i = Mis<B>I</B>nterpretation/confusion/human errors at a man-system
<B>I</B>nterface
<P>m = Hardware <B>M</B>alfunction attributable to system deficiencies, the
physical environment, acts of God, etc.
<P>M = <B>M</B>alfunction specifically due to electronic or other interference
<P>+ = Beneficial; - = problematic with none of the above categories
<P>@ = This item is also listed in another category
<H1><A NAME="3">Items Listed by Categories</A></H1>
<H2><A NAME="4">Recent Items (yet to be merged in)</A></H2>
Vm
PGN's Univ. Maryland survivable systems course beset with survivability
problems (S 25 1:)
<P>fm
California government agencies' computers fail, cars impounded;
Pac*Bell blamed (R 20 62)
<P>Vfm
Software disaster leaves new Australian submarine unfit;
wide range of pervasive hardware/software failures reported (R 20 48)
<P>*fh
Cancelling errors, serendipity in avoiding risks, and Kepler:
note by Henry Baker (R 20 48,51)
<P>Vm
Weather-predicting Cray C90 supercomputer lost in
fire, weather predictions reduced (R 20 62)
<P>Vm
Netcom file-server hardware outage loses half of the e-mail
customers, depending on first letter of name (R 20 49)
<P>m
Airport security check powers up computer (R 20 55)
<P>f
MIT system weather command gives "Temp: 2147483647 F
(2147483647 C)"; yes, that's <I>2<sup>31</sup> - 1</I> (R 20 51-52);
<I>USA Today</I> weather page: high of 577 F (R 20 58)
<P>?
NOAA radio +61 degrees F, wind-chill -64 (R 20 57)
<P>f/m/h?
Date failure on weather.com: 28 Apr not 16 Sep (R 20 58)
<P>$fh
Dispute over membership software used by German SPD (R 20 41)
<P>$f
Reliability of NT in embedded applications (R 20 41)
<P>f
Microsoft Word footnote problems irks federal appeals court:
Word does not count words correctly (R 20 52)
<P>fe
IE2 cannot read www.microsoft.com for upgrade (R 20 55)
<P>if
More anomalies in Microsoft driving directions (R 20 62-63)
<P>?
Risks of "self-destructing e-mail" from Disappearing Inc.
(R 20 62)
<P>h,h
Linux banned after Samba misconfigation blocks NT authentication (R 20 61)
<P>rh
Vending machine default phone number 000 (Australian emergency number)
yields hundreds of false alarms (R 20 47)
<P>f
Kangaroo helicopter responses mess up Australian virtual-reality simulators
(R 20 47,76)
<P>hf
Can you trust AT&T Wireless PCS text messaging? (no)
(R 20 52-53)
<P>f
Risks of financial planning engines with bogus results
(R 20 48)
<P>h
EverQuest is the "digital version of crack" (R 20 52)
<P>rd
More on California's software woes: welfare system problems
(R 20 53)
<P>h (but blamed on computer)
Argos retail offered Sony Nicam TV for 3£ instead
of 300£ (R 20 57)
<P>f
E*Trade Market Watch shows Dow Jones average at $1,
down $10936.88 (R 20 56)
<P>Vmfm
Power coming back on causes UPS to lose power (R 20 55)
<P>+/-??
Programming competency and the use of FORTH (R 20 49-53)
<P>$f
Toyota smog-warning computer suit (R 20 48)
<P>fh
CNN report on Gary Shandling lawsuit
names him as "Changeling"; spelling corrected? (R 20 47)
<P>h White House admits over one year of VP's e-mail lost forever (S 25 4:8,
R 20 91)
<P>f Software fault stops 76,000 customers receiving phone calls (S 25 4:8,
R 20 87)
<P>h UCITA, the Uniform Computer Information Transactions Act (Schneier,
S 25 4:8 and R 20 87, and Simons in August 2000 <I>CACM</I> Inside Risks)
<P>f Microsoft Explorapedia Nature: earth rotates in wrong direction (R 20 87)
<P>+ Patent office revamps Web patent reviews (R 20 87)
<P>h-
50 million U.S. adults at risk for Internet illiteracy (R 21 08-09; S 26 1:18)
<P>(f/m/h?)
Computer-related sewage release into Massachusetts Bay
(R 21 08; S 26 1:18)
<P>fid
Not-so-smart weapons in Kosovo (R 21 01; S 26 1:18)
<P>de
Satellite system outage hits Associated Press (R 21 04; S 26 1:18)
<P>m/f/h?
Root servers used by Network Solutions failed (R 21 03; S 26 1:18-19)
<P>$dh
$35M San Mateo CAlifornia health system upgrade is a downer;
receivables backlog over $40M; blame scattered (R 20 98)
<P>$d
WA King County blew $38 million on canceled payroll system (R 21 01; S 26 1:19)
<P>eh
Northeastern University admits 25% too many students (600 extra) after
DB upgrade loses potential applicants (R 21 01; S 26 1:20)
<P>f CSX crew spots problem signal, averts collision; insulation
problem? (R 21 04; S 26 1:20)
<P>e
Upgrade to Guildford Station (Surrey, UK) software
disables hundreds of train tickets for automated gates (R 20 94: S 26 1:20)
<P>fi
DC Metro can't label rerouted holiday trains on 4 Jul 2000: confusion
(R 20 95; S 26 1:20)
<P>m
Computer crash caused loss of scheduled cab pickups (R 20 98; S 26 1:20)
<P>m
Sliced fiber-optic cable in Lancaster PA disrupts local and long-