RISKS-LIST: RISKS-FORUM Digest Wednesday 20 September 1989 Volume 9 : Issue 26 FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator Contents: Hospital problems due to software bug (Joe Morris) Man-Machine Failure at 1989 World Rowing Championships (Geoffrey Knauth) Responsibility, Doctors, Military vs Software Developers (Leslie DeGroff) Organizational Accreditation: More Thoughts (Frank Houston, Jon Jacky) An interesting answer to the distributed time problem (Roy Smith) Re: Risks of distributed systems (D. Pardo) The RISKS Forum is moderated. Contributions should be relevant, sound, in good taste, objective, coherent, concise, and nonrepetitious. Diversity is welcome. CONTRIBUTIONS to RISKS@CSL.SRI.COM, with relevant, substantive "Subject:" line (otherwise they may be ignored). REQUESTS to RISKS-Request@CSL.SRI.COM. TO FTP VOL i ISSUE j: ftp CRVAX.sri.comlogin anonymousAnyNonNullPW cd sys$user2:[risks]get risks-i.j . Vol summaries now in risks-i.0 (j=0) ---------------------------------------------------------------------- Date: Wed, 20 Sep 89 09:54:11 EDT From: Joe Morris (jcmorris@mitre.arpa) Subject: Hospital problems due to software bug >From the _Washington_Post_, Wednesday, 20 September, page F-1 (the separate business section), without permission... SICK SOFTWARE CHECKS IN AT 100 HOSPITALS Programming Error Affects Admissions About 100 hospitals around the country, including Washington Hospital Center, were forced yesterday to switch from computers to pen and paper for major bookkeeping functions because a software program could not figure out what day it was. Officials said there was no permanent loss of data or threat to treatment of patients. But the incident, apparently caused by a mistake in programming, demonstrates how institutions are accepting the risk that major disuptions might occur in the workplace as more and more functions are handed to computers. The incident affected hospitals that use software and services provided by a Pennsylvania company called Shared Medical Systems Corp. The company stores and processes information for hospitals on its own mainframe computers and provides software that can be used on IBM equipment. Problems began to appear at numerous hospitals early yesterday morning. As call after call for help arrived at SMS headquarters, technicians there realized a pattern was emerging and advised clients to shut down parts of their computer systems as they searched for the cause. The problem was traced some hours later to a program that allows hospitals to automate the ordering and reporting of laboratory tests. Due to a fault in the aging software, the machines were unable to accept as valid the date September 19, 1989, and went "into a loop," refusing to work, spokesman A. Scott Holmes said. By day's end, computer services at about 100 of SMS's 600-700 client hospitals had been disrupted. One computer specialist described the problem as a "birth defect", an accidental fault put in a program in its early days that later threatened the system's health, in contrast to a "virus," a program that is written with the deliberate purpose of replicating itself and causing disruption. The incident also underlined how exacting the computer programmer's art can be. To build a system that handles routine tasks of hospital administration, programmers must write thousands of lines of "code," or computer instructions, that are intended to let the machine know what to do in every conceivable circumstance. At Washington Hospital Center yesterday, nurses used paper forms to admit patients, saving the information for entry when the computer system was restored. Staff members also had to manually request lab tests because the automated system would not work. Delays caused inconvenience but "patient care was not affected," said Claire Fiore, directory of public affairs and marketing for Washington Hospital Center. By about 5 p.m., the hospital's admissions computers were up and running again. Also hit was Capitol Hill Hospital, which reported minor disruptions. "We were able to function normally," said spokeswoman Lisa Poulter. "It may just take a little longer to order services." ------------------------------ Date: Tue, 19 Sep 89 10:27:09 EDT From: geoff@sunfs3.UUCP (Geoffrey Knauth) Subject: Man-Machine Failure at 1989 World Rowing Championships In the Petite Finals of the 1989 World Rowing Championships in Bled, Yugoslavia, five strokes into the Coxed Fours race the microphone wire came loose from my cox-box (an amplifier and tiny computer). I lay in the bow and coxed the entire race unaware that the speaker in the stern was out. Some of my crew could hear me, some could not, and in the confusion we rowed vacantly, far below our potential. The truth of a loose wire discovered just after the finish, that I had not fastened the connector well enough, and that a year's worth of hard work and good speed had just been wrecked, contained enough pain and horror to burn in my soul forever. Geoffrey S. Knauth, Camex, Inc., 75 Kneeland St., Boston, MA 02111 (617) 426-3577 standard disclaimers [Risks are certainly pervasive. This problem is not life-or-death or even row-versus-wade, but nevertheless must have had a devastating effect on all those involved -- but particularly the cox. On the other hand, I am surprised that in a sport so close to being natural (apart from computer designed shells) a computer would be permitted on-board. Next we might have on-line sensors monitoring crew heart rates to probe their limits, automated instructions from the computerized coxswain replacement using digitally synthesized voice and ear-phones, and even remote computer guidance from the coach overhead in a helicopter. Technology knows no bounds! PGN] ------------------------------ Date: Mon 18 Sep 89 09:53:56-PST From: Leslie DeGroff Subject: Responsibility, Doctors, Military vs Software Developers A varety of Risks messages have discussed medical risks, military risks and software accreditation. A recent one from Douglas Jones triggers me to present a different perspective on the issues of risks and accreditations. A fundamental feature of the current medical systems including accredited institutions is that Doctors have a very high level of personal responsibility for what happens. It is not the institution on the line, it's Dr. XXX and in most larger institutions there are reviews of any "failure" that the Dr. must answer to peers that understand the issues quickly after such a failure. There are are drawbacks to this system, in many cases Dr's over compensate to this Command responsibility by disregarding or belittling input from patients or nurses with more contact with patients. In the military also there is a somewhat corrupted notion of command responsibility, officers involved in major incidents/accidents are brought to trial with both career and freedom on the line if there have been fatalities. The breakdown in the military system comes in the fact that the line officiers are often in catch 22, if they don't use an automated system they are risking their ship and if they do and it is used to attack the wrong targets they are still at risk. There are two different messages that come out of consideration of the medical and military systems, one that creator/providers be made more personally responsible and two that users themselves bear major responsibility for results. This would have a multiplicative effect on lowering risks, personal responsibility (ownership) of creators tends to drive craftmanship and quality and usage responsibility tends to slow down the adoption of risk edge tools. Les DeGroff (intellicorp support) ------------------------------ Date: Tue, 19 Sep 89 11:30:45 -0400 From: houston@itd.nrl.navy.mil (Frank Houston) Subject: Organizational Accreditation for Computer Assurance: More Thoughts In hope of stimulating more thought and discussion, I submit the following responses to comments about my "food for thought" message. >Date: Fri, 15 Sep 89 08:51:05 CDT >From: Douglas W. Jones >Subject: Medical accreditation: good for big shops only? >... >One primary difference, from the accreditation point of view, is that most >hospitals are big, while many quite competent software shops are small. >Hospitals with fewer than 100 employees are rare, while there are many >corporations in both the software and hardware fields that are smaller. I suggest that Dr. Jones look around his home state and the neighboring states. Many hospitals are not large, and university hospitals are exceptionally large compared with hospitals in general. It may be true that hospitals with fewer than 100 employees are rare, but they are held to standards like JCAHO's to receive Medicare and insurance payments. Moreover, JCAHO accredits many other organizations and facilities, such as nursing homes, which are small. >Another difference is that most hospitals have only a small number of staff >physicians. Most physicians practicing in a hospital are not on the staff, >but instead, have an associate relationship. Good point. It is also true that many software and engineering shops supplement their staff with consultants, which sets up a similar situation. Nevertheless, I believe that JCAHO reviews credentials and records of both staff and associates when considering renewal of accreditation. The task of reviewing staff credentials for an engineering firm should not be any more difficult. >Finally, a hospital is a clearly defined organizational unit; even if it is >part of a hospital chain, the physical and staff boundaries are easy to. >define. Many software organizations are far harder to circumscribe. Finding >accreditable units in a large corporation may be quite difficult, and I doubt >that entire corporations (say Ford or Xerox) are appropriate units of >accreditation. I do not suggest that accreditation will be an easy concept to implement. Consider, however, the impact of a customer requirement that an organization be "accredited for the development of critical software/systems." I am sure that Ford or Xerox would find a way to identify the appropriate units if FAA made accreditation a prerequisite for award of ATC contracts. >These comments are reasonably representative of a fairly extreme libertarian >view that government regulation of the free market is inherently inept and has >a corrupting influence on the quality of services offered by the market. I must point out that my proposal does not specify accreditation by a government agency. In fact, having close up and personal experience with government regulation, I share some facets of the "libertarian view." JCAHO is not a government agency; it is more like an industry consortium; another reason why I suggested it as a preliminary model. >Government did not invent the Computing Sciences Accreditation Board, and it >did not invent the Institute for the Certification of Computing Professionals. >Right now, nobody is forced to seek accreditation or certification, but it is >not hard to imagine a few court cases establishing the principle of strict >liability for losses caused by software failures, and if this happens, I see >little hope for avoiding forced adherence to some kind of accreditation or >certification standards in the software industry. If government does not >force these on us, the insurance industry that emerges to provide malpractice >insurance for programmers will do it. I believe that government would be quite willing to defer to a private sector institution for regulating developers of critical software and systems, as it has in the case of JCAHO for medical practice. Product liability underwriters and other insurers probably would also. >Date: Wed, 13 Sep 89 10:46:37 PDT >From: Bob Ayers >Subject: Medical accreditation: based on "customer" clout? >... >First, it's not that Medicare accepts accreditation as quality-proof, but >will accept real proof too -- rather they accept (as I understand it) _only_ >accreditation. Legally Medicare and, therefore, Medicaid accept JCAHO accreditation in lieu of their own reviews. JCAHO accredited hospitals are "deemed" to be in compliance with most of the Medicare Conditions of Participation for Hospitals. I suggest that both individuals and organizations need to be accredited. If you look carefully at my outline, I suggest that an accreditation board might examine professional certifications. Nancy Leveson has argued very persuasively that, paraphrasing, "...someone competent must bear responsibility for software in critical systems ... some sort of professional licensing or certification is needed." I have been skeptical, but now I think that, with the further incentive of accreditation, professional certification could be made to work. I do not think accreditation is a "silver bullet." It is just as corruptible as any system of standards; however it uses the power of the marketplace to make corruption counterproductive. Both industry and customers would soon mistrust a corrupt accreditation system. The views expressed above are my own and do not reflect in any way the policies of FDA. Frank Houston ------------------------------ Date: Fri, 15 Sep 1989 16:50:58 PDT From: JON@GAFFER.RAD.WASHINGTON.EDU (Jon Jacky) Subject: Organizational Accreditation (What is BS5750/ISO9001?) In RISKS 9(23), Frank Houston (houston@itd.nrl.navy.mil) proposed consideration of some kind of organizational accreditation for software quality --- the idea was, the whole organization would be certified, rather than particular people or products. Does something like this already exist in the UK? Inside the front cover of SOFTWARE ENGINEERING 88, PROCEEDINGS OF THE SECOND IEE/BCS CONFERENCE, there is an ad from a systems house bearing an official-looking seal that says "BSI - REGISTERED FIRM". The ad copy says, "First Systems House to Win BS5750 (now ISO 9001)", without further explanation. Can any British RISKS readers enlighten us? - Jon Jacky, University of Washington ------------------------------ Date: 7 Sep 89 00:05:09 GMT From: roy%phri@uunet.UU.NET (Roy Smith) Subject: An interesting answer to the distributed time problem Lots of things in lots of places depend on having a consistant time reference distributed to many places. Lots of people have worked out high-tech solutions to the problem. Most of them don't work very well. When's the last time you saw a master clock system that worked? The one in our building certainly doesn't. Well, today, I saw an interesting solution in Mt. Sinai Hospital in New York. Take any of the thousands of closed circuit TVs in the hospital and set it to channel 6 and you get a picture of a clock. And not a digitized, computer generated picture either. Somewhere there is a TV camera pointed at a good old sweep-secondhand analog clock, and that's what you see on channel 6. Sometimes low-tech solutions are the best. Roy Smith, Public Health Research Inst., 455 First Avenue, New York, NY 10016 ------------------------------ Date: Sun, 17 Sep 89 12:05:06 PDT From: pardo@june.cs.washington.edu Subject: Re: Risks of distributed systems In RISKS 9.25, Eugene Miya writes (about the special risks of parallel and distributed systems) that ``[I]t isn't the researchers ... who nead exposure [to parallel/distributed issues], it's the students who need this [...]''. There are a growing number of schools that are using Ada to replace Pascal and Modula-2 in the introductory programming courses. Using Ada gives a standardized mechanism for introducing issues such as exceptions and (concurrent) tasks. I think that everybody will agree that distributed/parallel is under-represented, under-standardized, and over-risky. There is hope, however. ;-D on ( Is parallel uniprocessor an oxymoron? ) Pardo ------------------------------ End of RISKS-FORUM Digest 9.26 ************************