Subject: RISKS DIGEST 10.30
REPLY-TO: risks@csl.sri.com

RISKS-LIST: RISKS-FORUM Digest  Tuesday 4 September 1990   Volume 10 : Issue 30

        FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS 
   ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Contents:
  Business Week on High-Tech Amusement Parks (Karl Lehenbauer)
  Arabian heat causing problems with US weapons computers (Jon Jacky)
  Re: Stonefish mine (Chaz Heritage via Richard Busch)
  Flight simulator certification (Henry Spencer)
  Glass cockpits (Martyn Thomas)
  "Wild failure modes" in analog systems (Kent Paul Dolan)
  Faultless Software (Robert L. Smith)
  Comment on Software Reliability Synopsis (Martin Minow)
  Database searches and counseling center confidentiality (Derek Beatty)

The RISKS Forum is moderated.  Contributions should be relevant, sound, in good
taste, objective, coherent, concise, and nonrepetitious.  Diversity is welcome.
CONTRIBUTIONS to RISKS@CSL.SRI.COM, with relevant, substantive "Subject:" line
(otherwise they may be ignored).  REQUESTS to RISKS-Request@CSL.SRI.COM.
TO FTP VOL i ISSUE j:  ftp CRVAX.sri.com<CR>login anonymous<CR>AnyNonNullPW<CR>
cd sys$user2:[risks]<CR>GET RISKS-i.j <CR>; j is TWO digits.  Vol summaries in 
risks-i.00 (j=0); "dir risks-*.*<CR>" gives directory listing of back issues.
ALL CONTRIBUTIONS ARE CONSIDERED AS PERSONAL COMMENTS; USUAL DISCLAIMERS APPLY.
The most relevant contributions may appear in the RISKS section of regular
issues of ACM SIGSOFT's SOFTWARE ENGINEERING NOTES, unless you state otherwise.

----------------------------------------------------------------------

Date: 3 Sep 90 11:18:30 CDT (Mon)
From: karl@sugar.hackercorp.com (Karl Lehenbauer)
Subject: Business Week on High-Tech Amusement Parks

In the September 10, 1990 issue of Business Week, a sidebar to an article
about problems at Universal Studios Florida ("MCA May Have Created A
Monster") is about high-tech amusement park rides.  The cavalier attitude
toward risks is startling.

The article reports that "bigger thrills are needed to lure a jaded
generation raised on dazzling movie special effects" and that, consequently,
Universal Studios spent over $200 million for rides such as "a giant
robotic shark that attacks a boatload of tourists, a three-story animated
King Kong, and a bone-jarring imitation earthquake."  The article also
notes that other theme-park operators, including of course Walt Disney Co.,
have "joined the race for ''animatronics,'' robots, and other computerized
contraptions."

	Yet as the bugs plaguing Universal show, technology has its
	price.  Says Joseph B. McHugh, vice-president of Ride & Show
	Engineering Inc., which made the Jaws and Earthquake rides:
	"The complexity of the systems means there are more components
	that can shut a ride down."

	The scariest prospect for a park operator is a Jaws that
	doesn't bite.  And all it takes is one software bug.  At
	Universal, the trick is to synchronize a moving tram and
	an animated shark or gorilla that runs on a fixed program.
	"If they're not coordinated exactly, they run into one
	another and parts get bent," says Q. David Schweninger,
	chief of Sequoia Creative Inc., Kong's creator.  Universal's
	rides are "way out in front of everyone else," he says.
	"The price is that you're going to have teething problems."

Later, the sidebar quotes Schweninger as saying "There are lessons
to be learned here," and adding that ride makers *may* insist on more 
shakedown time before new rides open.

	But none of this is likely to halt the shift to high-tech
	rides, which promise more safety and eat less real estate.
	Changing demographics also favor gentler high-tech rides
	over old-style gut-wrenchers such as roller coasters.

The article concludes by pinpointing the Disney Star Tours ride in
1987 as the first of the new wave of participatory rides and that
ride makers are working on dozens of variations of the simulator
ride for theme parks, casinos and special theaters.

Although it's reasonable to assume gentler rides would be safer than "old-
style gut-wrenchers," the claim seems to be that high-tech makes the rides 
safer, which the rest of the article seems to refute.  Oh well.

uunet!sugar!karl  Usenet access: (713) 438-5018

------------------------------

Date:    Tue, 4 Sep 1990 10:35:49 PDT
From: JON@GAFFER.RAD.WASHINGTON.EDU   (Jon Jacky)
Subject: Arabian heat causing problems with US weapons computers

Here are excerpts from a story in THE SEATTLE TIMES, Sept 3 1990, p. A8:

US TROOPS ALREADY UNDER ATTACK FROM SUN AND SAND by Molly Moore, 
Washington Post

... Patriot missiles, which are to protect critical military sites from attack
by Iraqi Scud-B missiles and attack planes, are controlled by computer
equipment housed in air-conditioned vans.  But the heat is so intense on each
van's metal shell that it raises the temperature inside.  

"Every now and then there is a glitch that makes the (radar) scope look blank,"
said one missile technician.  He said in an attack, the computers are supposed
to override any radar screen malfunction or other problem that might hinder
a human operator, and track the incoming missile on their own.  ...

... If the A-10 attack plane fights Iraqi tanks, it will depend on cylindrical
pods under its wings to jam the signals of enemy air defenses.  But weapons
loaders here said the heat renders the jammers useless after about one hour of
operation.  A typical sortie against hostile tanks would likely require far
more than an hour's protection ...

- Jon Jacky, University of Washington, Seattle jon@gaffer.rad.washington.edu

------------------------------

Date: 	Tue, 4 Sep 1990 10:03:21 PDT
From: Richard_Busch.sd@Xerox.com  
Really-from: Chaz Heritage
Subject: Forwarding: Stonefish mine

Chaz Heritage has requested that I forward the following as a candidate for
inclusion in the Risks Digest. He is apparently unable to get mail to you
directly.

richard
  - - - - - - - - - - - - - - -
From: chaz heritage:wgc1:RX
Sender: chaz heritage:wgc1:rx
Subject: Stonefish mine
Date: 3-September-90 (Monday) 4:34:36 PDT

In RISKS-FORUM Digest  Wednesday 29 August 1990   Volume 10 : Issue 26 Pete
Mellor asks a number of questions about the Stonefish mine which may be
possessed by Iraq:

>1. If the Iraqis have the software for a 'limited number' of mines, why
    haven't they got enough for an unlimited number?<

To the best of my knowledge each Stonefish is intended to have a unique
identity to allow selective arming or disarming. Therefore the software
installed in each mine will be slightly different from the others'. It is
likely that Stonefish's software is in ROM, and not loaded into RAM from
floppies as Mr. Mellor seems to suggest. Manufacturing more ROMs, even if
one knows how to assign new identities, would require a ROM-burner and a
supply of blanks. The sort of ROMs used are possibly 'strategic goods' and
not available directly to Iraq, nor, perhaps, to Cardoen.

>2. How does Stonefish 'hide' from a minesweeper?<

Gee, that sure is classified!

Minesweepers can detect mines by a number of methods, to each of which
countermeasures may be available. Sonar, for example, may be spoofed by
returning an amplified signal, too big for a mine (so it looks like a whale
or wreck, perhaps?). Countermeasures may exist against magnetic anomaly
detection, but these are not disclosed in the unclassified literature
(since MAD is used to detect SSBNs this is hardly surprising). There may
well be other countermeasures included in Stonefish's suite.

>3. How reliably can Stonefish identify ships by their engine noise
signature?
    What happens if your cruiser's big ends are rattling?<

The noise signature is mainly a function of the screw design, which is why
there was so much fuss over the Japanese selling the USSR some CNC
machining equipment capable of manufacturing low-noise screws. A ship
trying to avoid acoustic detection will proceed slowly and as quietly as
possible, but it cannot conceal the characteristic noise caused by screw
cavitation at any speed above a knot or two.

>4. Does Stonefish rely on some sort of sonar transponder
    to distinguish friend from foe?<

I imagine not, since if the mine were to transmit sonar in order to trigger
transponders located on friendly ships then it would render the mine very
susceptible to detection and countermeasures.

> 5. What are the chances that Iraq already has the software?<

100% chance of posessing the basic software if they possess the originally
issued mines. Each mine would be supplied with it. Reverse-engineering it
is probably a matter of copying a number of unusual boards and devices, and
would depend upon posessing at least one working example and sufficient
parts. Circuit boards used in weapons systems are often of unusual shape
and construction and replication of them would probably not be particularly
easy.

The target-recognition software, on the other hand, would probably consist
of digitised acoustic samples for comparison with the signature of the
target. If the mines were to be used against US or UK warships then samples
of their signatures would be required. These would probably not be
forthcoming. Any type of ship an example of which had been sold to Iraq
might possibly be at risk (assuming the Iraqis to possess the sampling
apparatus and the ability and time to use it), as might any types possessed
by navies considered at any time to be hostile to the UK Royal Navy.

> 6. The sophistication of Stonefish's recognition system argues for some
kind
    of artificial intelligence. If it's that smart, would it know who was
    winning and change sides accordingly?<

Personally I wouldn't consider Stonefish to be an AI. I don't think the
problem posed is much of a risk to Stonefish operators....

>7. Isn't it time that Jane's produced 'All the World's Software'?<

Yes, but it would be a far slimmer volume than their usual anti-bookshelf
masterpieces (I had to get rid of all my old ones before my upstairs bedsit
suddenly turned into a downstairs bedsit). I don't honestly think enough
information would be disclosed to make it worthwhile.

>The implication is not that Stonefish has been sold bundled to Iraq, but
enough technical information is in dubious hands for the Iraqis to have a
good go at building a look-alike<

Personally I wouldn't back them to do it. If I were a naval commander I
should consider the threat of anti-ship missiles fired from the air or from
the Kuwaiti islands against targets acquired by aircraft possibly still
within Kuwaiti airspace to be a greater threat than that of Iraq managing
to copy a sophisticated underwater weapons system to a deadline.

If, on the other hand, Carlos Cardoen is telling fibs (which would not
perhaps be entirely out of character) then it's possible that he's sold the
Iraqis a few Stonefish already. If so, it seems unlikely to me that they'll
work properly without reprogramming for the target signatures of US and UK
shipping.

Incidentally, the classification >'UK restricted: commercial in
confidence'< is among the lowest available. Almost all documents used by
the armed services, including such things as reminder cards carrying the
NATO phonetic alphabet, are classified 'restricted'; the term 'commercial
in confidence' is applied merely in the fond hope that it will be
respected, since it has only commercial meaning. The >'Technical
Description and Specification' of Stonefish< may well be no more than the
sort of thing small boys (like me) fill their carrier bags with every other
year at Farnborough - I have collected stuff in the past on classified
systems like JP233, also deployed in the Gulf. What these documents do not
disclose is the true performance of the system and its strengths and
weaknesses. That sort of information, operationally significant, is usually
classified at a higher level (within NATO probably 'Secret').

Chaz                                          [All disclaimers apply.]

------------------------------

Date: Tue, 4 Sep 90 13:16:21 EDT
From: henry@zoo.toronto.edu
Subject: Flight simulator certification

>> Does anyone know the certification requirements for simulators?)
>... I suspect there is nothing to cover
>simulators, since they are not directly involved in controlling flight.

There *are* certification requirements for simulators when they are involved in
pilot training, and in particular when used as substitutes for certain types of
in-flight training.  However, I think the emphasis has been more on precision
than on accuracy, i.e. on the sophistication and smoothness of the simulation
more than on its exact correspondence to reality.  Much is made, for example,
of the quality of the visual imagery provided.  I'm not sure how much has been
done on verifying faithful (as opposed to merely plausible) simulation of
behavior in obscure corners of the system.
                                          Henry Spencer at U of Toronto Zoology
                                          henry@zoo.toronto.edu   utzoo!henry

------------------------------

Date: Mon, 3 Sep 90 15:49:50 BST
From: Martyn Thomas <mct@praxis.co.uk>
Subject: Glass cockpits

The UK has a "confidential human factors incident reporting programme",
run by the RAF Institute of Aviation Medicine, to allow aircrew to report
incidents which may reflect badly on their competence, so that others may
learn from them without any risk of the crewmember who made the report
suffering any penalty. They have a magazine - FEEDBACK - and the last issue
publicised the failure of most of the automated systems on an approach to a
UK airport in a storm last February. (Reported in RISKS, I believe).

This issue carries a questionnaire asking flight crew for their opinions of
automation. 78 questions of the form (ring the appropriate number)

"if the automatics fail it	1 2 3 4 5 	if the automatics fail it
is always apparent				is never apparent "

The results of the survey will be *very* interesting. The survey is being
carried out on behalf of the UK Civil Aviation Authority. It is not clear
whether the results will be made public.

Martyn Thomas, Praxis plc, 20 Manvers Street, Bath BA1 1PX UK.
Tel:	+44-225-444700.   Email:   mct@praxis.co.uk

------------------------------

Date: Sun, 2 Sep 90 18:59:45 GMT
From: xanthian@zorch.sf-bay.org (Kent Paul Dolan)
Subject: "wild failure modes" in analog systems

>From: Pete Mellor <pm@cs.city.ac.uk>

>Synopsis of: Forester, T., & Morrison, P. Computer Unreliability and
>Social Vulnerability, Futures, June 1990, pages 462-474.

>In contrast [to digital computers], although analogue devices
>have infinitely many states, most of their behaviour is
>*continuous*, so that there are few situations in which they
>will jump from working perfectly to failing totally.

Unless my understanding from readings in Chaos Theory is entirely flawed, the
second sentence is simply false; it is now well known that analogue devices can
also (through design infelicities or just the perverseness of the universe) do
inherently "wild" state switches.  The classic example is the simple dribble of
water from a faucet, which, in the absence of analogue catastrophes, would be a
steady stream, or an equally spaced series of droplets, but is instead a series
of droplets whose size and spacing is unpredictable except statistically.

More important, this is now understood to be the _usual_ state of affairs, not
the anomalous one, in dealing with realistic analogue systems.

So, if the original authors' intent in demeaning our increasing
reliance on (possibly "un-failure-proofable") digital systems is
to promote a return to the halcyon days of analogue controls,
this is probably misdirected by the time the controls approach
the order of complexity of operation of the current digital ones.

We may just have to continue to live with the fact, true throughout
recorded history, that our artifacts are sometimes flawed and cause
us to die in novel and unexpected ways, and that we can only do our
human best to minimize the problems.

Just an observation in passing.

Kent   <xanthian@Zorch.SF-Bay.ORG> <xanthian@well.sf.ca.us>

------------------------------

Date: Tue, 4 Sep 90 13:25:43 -0400
From: Robert L. Smith <rlsmith@mcnc.org>
Subject: Faultless Software

    The recent discussion in RISKS of the need for "ultrareliable" --
i.e., faultless -- software and the impossibility of obtaining it has
been interesting, but lack of it is no compelling reason to prohibit
computers from life-critical service.
    Advocates of that conclusion forget the reliability advantage
software has over hardware and people system components, which is that
once a software bug is truly fixed, it stays fixed!  In contrast
consider the many times you repair hardware only to see it fail again
from the same cause, and coach people to do it right only to hear
they've forgotten later.
    Legal restrictions on software applicability would delay quality
improvements.  They would inhibit progress toward systems that truly
are safer than those whose logic elements reside solely in human
brains.  Software in life critical environments is like old age:  to
understand its desirability, one has only to consider the alternative.
    The question is, have more people died in life critical environ-
ments since software was installed than before, per man-hour of use?
If the answer is no, then the argument is specious.  Even if it is
yes, which I doubt, that is reason only to intensify testing and
debug.  Software engineering has not yet built all the tools
conceivable to that end.

Regards, rLs

------------------------------

Date: Mon, 3 Sep 90 20:27:01 PDT
From: "Martin Minow, ML3-5/U26  03-Sep-1990 2253" <minow@bolt.enet.dec.com>
Subject: Comment on Software Reliability Synopsis

Thanks to Pete Mellor for posting
>Synopsis of: Forester, T., & Morrison, P. Computer Unreliability and
>Social Vulnerability, Futures, June 1990, pages 462-474.
> Conclusion (quoted):
>Accordingly, we recommend that computers should not be entrusted with
>life-critical applications now, and should be only if reliability improves
>dramatically in the future.
 
There is a risk/benefit that needs to be examined.  If we remove computers
from, say, hospital intensive care units because they are "unreliable" will
we save lives that might be killed by an errant computer or kill others whose
lives might have been saved by that same unreliable system.

>Given the evidence presented here, several ethical issues also emerge for
>people in the computer industry. For example, when programmers and software
>engineers are asked to build systems which have life-critical applications,
>should they be more honest about the dangers and limitations?

Does the article really claim that engineers who know they are building
life-critical applications are not honest about the dangers and limitations?
My experience has been that people are quite aware of their responsibilites.
On the other hand, systems that were never designed to be life-critical are
often used in unexpected ways.  Consider a speech synthesizer designed as a
speech aid for a disabled person.  While not designed as a "life-critical"
system, it IS the voice that that person must use to call for help.  What
is an ethical professional to do?  Refuse to build a speech synthesizer as it
*might* be used in a life-critical situation and the technology doesn't
yet allow us to design a perfect synthesizer?

>Are computer
>professionals under an obligation if a system fails: for example, if a patient
>dies on an operating table because of faulty software, is the programmer guilty
>of manslaughter, or malpractice, or neither?

Who cares?  Will charging programmers with manslaughter really yield better
quality software?  This seems like exactly the wrong thing to worry about.

>How is it that the computer industry
>almost alone is able to sell products which cannot be guaranteed against
>failure?

Because people buy the stuff.  Some software, by the way, *is* warrented
against failure.  All the type of warranty does is affect the price and
time to market.  I rather doubt that it effects the actual quality.

Martin Minow          minow@bolt.enet.dec.com

------------------------------

Date: Tue, 04 Sep 90 12:23:42 EDT
From: Derek.Beatty@COSMOS.VLSI.CS.CMU.EDU
Subject: Database searches and counseling center confidentiality

   Here's a minor variation on an old theme:

   The Carnegie Mellon's online library catalog includes a full-text
database of the faculty/staff directory, and can be used to look up
anyone at CMU given their telephone extension.  This brings to light
a problem with confidentiality and the university's counseling
center.  If the counseling center phones a student and must leave a
message, they leave only their telephone number and receptionist's
first name, to protect against any stigma that might be associated
with seeking their services.  Easy access to the reverse telephone
index function via database searching erodes this effort at
confidentiality even though the library database publicizes no new
information (it's all in the published hard copy directory).
   Here again a large quantitative change (in lookup time) introduces
qualitative differences (it becomes plausible that a roommate might
snoop out of idle curiosity).  Awareness of this might lead someone,
feeling lowly as a (internet?) worm, to forego psychological services.
 
  -- Derek Beatty,      grad student, CMU Computer Science

------------------------------

End of RISKS-FORUM Digest 10.30
************************