9-Nov-85 00:00:22-PST,23415;000000000001 Mail-From: VONHENKE created at 8-Nov-85 23:58:10 Date: Fri 8 Nov 85 23:58:10-PST From: RISKS FORUM (Peter G. Neumann, Coordinator) Subject: RISKS-1.22 Sender: VONHENKE@SRI-CSL.ARPA To: RISKS-LIST@SRI-CSL.ARPA RISKS-LIST: RISKS-FORUM Digest Wednesday, 16 Oct 1985 Volume 1 : Issue 22 FORUM ON RISKS TO THE PUBLIC IN COMPUTER SYSTEMS Peter G. Neumann, moderator Friedrich von Henke, guest moderator Contents: Administratrivia (Friedrich von Henke) Medical software incidents (Nancy Leveson) European activities (Udo Voges) Robots are different (Jerry Saltzer) Automobile computer control systems (Bennett Smith) Police computers (Dave Dyer) Electronic Surveillance (Geoffrey S. Goodfellow / Bill Keefe) Network Mailer Woes (Lynne Moore) Databases, grades, etc. (Karl Kluge, Andy Mondore, Mark Sienkiew) Summary of Groundrules: The RISKS Forum is a moderated digest. To be distributed, submissions should be relevant to the topic, technically sound, objective, in good taste, and coherent. Others will be rejected. Diversity of viewpoints is welcome. Please try to avoid repetition of earlier discussions. (Contributions to RISKS@SRI-CSL.ARPA, Requests to RISKS-Request@SRI-CSL.ARPA) (FTP Vol 1 : Issue n from SRI-CSL:RISKS-1.n) ---------------------------------------------------------------------- Subject: Administratrivia From: Friedrich von Henke (vonHenke@SRI-CSL) I am temporarily acting as guest moderator of the Risks Forum. Peter Neumann had to go rather abruptly on an overseas trip, and the transition happened a bit disorderly. My apologies to all of you who were eagerly awaiting their twice-weekly cost of RISKS readings but had to go without. I hope to have things under control now. Apparently the hiatus has had the effect of slowing down the stream of contributions to a merely trickle; please don't hesitate to get the flow going again! Friedrich von Henke ---------------------------------------------------------------------- Date: 11 Oct 85 19:39:27 PDT (Fri) From: Nancy Leveson To: RISKS@sri-csl.ARPA Subject: medical software incidents I was just on a panel concerned with Software Safety at an IEEE conference on Computers in Medicine and heard about some more incidents involving software faults. The first was cited in a recent RISKS forum message (about the programmable implanted pacemaker which was inadvertently reprogrammed by emitted magnetic fields from an anti-theft device in a retail store), but the patient was cited as having survived. Unfortunately, his weakened heart was unable to stand the increased pace, and he died. Other recalls by the FDA involve: 1) An infusion-pump (used for insulin) had a software problem which caused the infusion of insulin or dextrose to be delivered at the maximum rather than the lower intended rate. This occurred when certain valid data was entered according to user instructions. 2) A programmable pacemaker "locked-up" when being reset by an external programming device. Luckily this occurred in a doctor's office, and the doctor was able to revive the patient. 3) A multiple-patient monitoring system was recalled because the software got patients' names mixed up with the wrong data. 4) An algorithm was incorrectly programmed in a diagnostic lab instrument which caused certain patient data to be reported erroneously as all zeros. The reference for these incidents (for those who want to quote them) is: H. Bassen, J. Silberberg, F. Houston, W. Knight, C. Christman, and M. Greberman. "Computerized Medical Devices: Usage Trends, Problems, and Safety Technology," in Proc. 7th Annual Conference of IEEE Engineering in Medicine and Biology Society, Sept. 27-30, 1985, Chicago, Illinois, pp. 180-185. Nancy Leveson University of Calif. Irvine ------------------------------ Date: Fri, 11 Oct 85 11:38:57 PDT From: Udo Voges To: RISKS@SRI-CSL.ARPA Subject: European activities I would like to bring some activities to your attention which are going on in Europe, especially within and triggered by EWICS TC 7. The European Workshop on Industrial Computer Systems (EWICS), TC on Systems Reliability, Safety and Security (TC 7) is working since about 10 years in this area, having some 100 members from industry, research and university. Previous work resulted in Position Papers on Development of safety related software Hardware of safe computer systems Guidelines for verification and validation of safety related software Guidelines for documentation of safety related computer systems Techniques for verification and validation of safety related software System requirements specification for safety related systems Current working areas include: System integrity Software quality assurance and metrics Design for system safety Reliability and safety assessment Besides conducting about four working meetings a year the TC is organizing the IFAC/IFIP Workshop on Achieving Safe Real-Time Computer Systems (Safecomp'79, '82, '83, '85, '86). The results of the work of TC 7 are introduced into the standardisation process (IEC, ISO, and national bodies) as well as used by companies and licensing authorities. Those interested in more information can either contact me or the current Chairman of TC 7: Mr. J.M.A. Rata, Electricite de France, 1 Avenue du General de Gaulle, F-92141 CLAMART FRANCE. There exists an American counterpart to EWICS TC 7, but it was not possible to attract enough interested persons to keep it alive. The Japanese counterpart is also active, but due to the language barrier communication is minimal. Udo ------------------------------ Date: Sat, 12 Oct 85 01:30 EDT From: Saltzer@MIT-MULTICS.ARPA Subject: Robots are different To: risks@SRI-CSL.ARPA When someone gets pinned to the wall by a robot, something different is going on as compared to when someone gets gunned down by an FBI agent operating under incorrect information retrieved from the NCIC. Both cases may lead to specific tragedies, yet the example of risks from robots seems to me to be qualitatively different from many other computer-use risks. The difference is that robots are used primarily in environments where mechanically-oriented people are accustomed to balancing the risks of new machinery against the benefits. These people have, over the years, learned to deal with risks from gear trains and drive belts, from swinging tailends on steamshovels, from runaway elevators, from inadequately supported cranes. They watch out, they learn from accidents, their insurers offer advice, they make mistakes and take risks, and they learn. To a first approximation, an industrial robot presents a risk similar in kind to other new machinery, and there is a moderately well-working system in place that is accustomed to watching for the risks. If anything, the average mechanic is suspicious of a new piece of machinery in direct proportion to its complexity, newfangledness, and gadgetry level, so is probably expecting the robot to screw up in marvelous ways. One might wish to argue with the particular balance that an industry has struck between risks and benefits, but it is unusual to find one in which mechanical risks are not understood at least moderately well. The mechanic's suspicion of the new gadget is the essence of what seems to be missing in many other applications of computers, and why it is so important to raise awareness of the need to assess risks. I'm not convinced we need to harass our colleagues in the robot business with risk-consciousness-raising. We should be instead talking to their installers to find out what we can learn. Jerry Saltzer ------------------------------ Date: Wed, 23 Oct 85 11:14:29 -0100 From: ircam!bks@seismo.CSS.GOV (Bennett Smith) To: NEUMANN@SRI-CSL.ARPA Subject: Automobile computer control systems susceptible to interference By chance I saw an article in an issue of the "Journal of Environmental Engineers" (published in England, date of issue about 10 months ago, I believe) about the sensitivity of a microprocessor-controlled automobile control system to external electromagnetic radiation. As I recall, a CB transmitter near the car could, at the right frequency, make the engine slow down or speed up. Perhaps this article would interest some of your contributors. Bennett Smith IRCAM 31, Rue Saint Merri 75004 Paris, France {seismo,philabs,decvax}!mcvax!ircam ------------------------------ Date: 15 Oct 1985 23:42:01 PDT Subject: The human element From: Dave Dyer To: risks@SRI-CSL.ARPA The human element really is where the action is, and it is a completely two-edged sword; Human actions which have the power to "fix" something almost inherently also give the power to "break" things equally severely. Conversely, weighty check and balance systems intended to prevent abuse end up preserving the status quo, however good or bad that may be. The "police computer horror story" I'm most familiar with is illustrative. This is a well documented case I've been reading about in ACLU publications. It seems some poor soul had his wallet stolen, and some criminal adopted his identity and later was involved in a robbery/murder. Through some circumstances peculiar to the case, the stolen identity, but not the culprit, were known to the LAPD. The detectives working on the case put the stolen identity into a national police computer. Our hero was stopped for a routine traffic citation, the computer coughed his name up, and he ended up on ice for a few days as a murder suspect. So far, this is pretty harmless and understandable. Eventually the guy's identity and and non-involvement were establishd and he was turned loose. Then it happened again. And Again. The guy began carrying a letter from the local chief of police, saying he wasn't the guy the computer said was wanted, but that didn't cut it when he traveled. The problem was that the LAPD detectives who put in the original "want" refused to remove it. Eventually the guy (and the ACLU) got the courts to mandate expunging the computer. I think the detectives involved and the LAPD are being sued. Quite rightly. The point is, it is <> to design a system that can do its intended job, permit discovery and correction of errors, and resist unautherized or inappropriate use. I can't imagine a system that can do all three. ------------------------------ Date: 24 Oct 1985 11:17-PDT Subject: Electronic Surveillance. From: the tty of Geoffrey S. Goodfellow [forwarded to RISKS by Bill Keefe from TELECOM Digest Volume 5, Issue 55, October 24, 1985] Americans' Privacy Exposed by New Technology, Congress Told By LEE BYRD - Associated Press Writer WASHINGTON (AP) - The explosion in communications technology has so outpaced privacy laws that Americans have little or no protection against a plethora of new ways for government or private adversaries to pry into their lives, a congressional agency reported today. The non-partisan Office of Technology Assessment found that 35 out of 142 domestic federal agencies use or plan to use various electronic surveillance methods, including modern devices not governed by a landmark 1968 law that circumscribed the use of wiretaps and bugs - concealed microphones. The agency said 36 agencies, not counting those in foreign intelligence, already use a total of 85 computerized record systems for investigative or intelligence purposes, and maintain 288 million files on 114 million people. The report raised the ''technically feasible'' specter of these being linked into a single data base network that could track untold numbers of citizens without due cause. The report, requested by House and Senate committees, noted that many new and uncontrolled methods of surveillance are made possible by the very technologies of which more and more Americans are availing themselves - electronic mail, computer conferencing, cellular and cordless telephones, beepers and electronic pagers. Intercepting such devices is easy, and ''the law has not kept pace,'' the agency said. But other devices, such as miniature television cameras and pen registers - which monitor the numbers called on a given telephone line - have enabled new ways to spy on people even if their own communications habits are more old-fashioned, the agency noted. Rep. Robert W. Kastenmeier, D-Wis., chairman of the House Judiciary subcommittee on courts and civil liberties, said the study ''shows how the law in this area has broken down; it is up to Congress to fix it. If we fail to act, the personal and business communications of Americans will not have the privacy protection they deserve.'' Sen. Charles McC. Mathias, R-Md., said the report ''documents how new and more intrusive forms of snooping have followed in the wake of the exciting advances in communications technology,'' and agreed Congress must ''bring federal privacy laws up to date.' Rep. Don Edwards, D-Calif., chairman of the House Judiciary subcommittee on civil and constitutional rights, said, ''While the attorney general of the United States is claiming that the civil liberties granted by the Constitution should be limited to the 'original intentions' of the framers, the technological possibilities for government surveillance have exploded. The framers knew nothing of closed-circuit television, wiretapping and computer data banks.'' The report noted that the Fourth Amendment, which protects ''the right of the people to be secure in their persons, houses, papers and effects, against unreasonable searches and seizures,'' was written ''at a time when people conducted their affairs in a simple direct, and personalized fashion.'' Neither, said the report, has Title III of the Crime Control and Safe Streets Act of 1968, which was designed to protect the privacy of wire and oral communications, kept pace. ''At the time Congress passed this act,'' the report said, ''electronic surveillance was limited primarily to simple telephone taps and concealed microphones. Since then, the basic communications infrastructure in the United States has been in rapid technological change.'' The congressional agency said it could not estimate the extent of electronic surveillance in the private sector, saying only ''it is probable that many forms ... go undetected, and if detected, go unreported.'' But in its survey of the federal bureaucracy, OTA found 35 agencies, mostly in the Justice, Treasury and Defense departments, used or planned to use: -Closed circuit television, 29 agencies. -Night vision systems, 22. -Miniature transmitters, 21. -Electronic beepers and sensors, 15. -Telephone taps, recorders, and pen registers, 14. -Computer usage monitoring, 6. -Electronic mail monitoring, 6. -Cellular radio interception, 5. -Satellite interception, 4. As for the 85 computerized record systems that could be used for surveillance purposes, none of the operators provided statistics requested by the OTA on record completeness and accuracy. Under the 1968 law, wiretaps and bugs are prohibited without a court order based on the affirmation of a high-ranking prosecutor that a crime has occurred, that the target of the surveillance is involved, and that other means of investigation would be ineffective. According to the Administrative Office of the U.S. Courts, federal and state judges approved 801 out of 802 requests last year for electronic surveillance, primarily wiretaps and hidden microphones, at an average cost of $45,000. The agency said that while there is some promise in emerging techniques for low-cost data encryption or other means to protect communication systems from eavesdropping, ''there is no immediate technological answer ... against electronic surveillance.'' Foreign intelligence cases are governed by a separate law, so the CIA, National Security Agency and Defense Intelligence Agency were not included in the survey. ------------------------------ Date: 0 0 00:00:00 CDT [sic! ed.] From: "UV2::MOOREL" Subject: Network Mailer Woes... To: "risks" In a recent issue of one of the digests on the net, there was a problem mentioned that seems to have a bearing on risks on computer systems, particu- larly as use of computer networking increases in the future. At their request, the names have been changed to preserve anonymity. Apparently for the past month, the people who reside on the bitnet have been unable to receive [this digest]. There is a long story behind this [...]. This story is also a *lot* of guesswork as to what happened. At the beginning of September, [our system] changed its host name to conform to the new domain name standards. The site we were using to get to bitnet, did not recognize the new name and began rejecting all mail from [our system]. We did not become aware of this because we were not receiving any rejections or errors back from the gateway. We were however, receiving mail *from* the people on Bitnet who were asking what happened to their [...] digest. We attempted to contact the people at [the gateway] but of course the mail failed and they never did anything to correct the problem which they, of course, were not aware of because nobody was complaining. (Sounds like a Catch-22 situation if I ever heard one). In any case, the problem has now been resolved. Unfortunately, these people have missed close to 50 digests. There is no way I can tie up the mailer at [either the host or the gateway] in order to remail the messages. I also understand that there is no way to use FTP from the bitnet. It appears that several incidents conspired together to cause the loss of this information, and although the loss was not critical, it will take much time and effort for the people involved to catch up. If this had been a more critical information transfer, it might have been corrected faster; however, there is a good chance that it would have gone undetected anyway. It seems to be one more reason for information about any potential changes to be passed on to any sites that may be affected and to be thoroughly checked on both ends to prevent this kind of thing from reoccurring. Lynne C. Moore (Moorel@Eglin-Vax.Arpa) ------------------------------ From: Karl.Kluge@G.CS.CMU.EDU To: risks@sri-csl.arpa Subject: Grade changing. Some doubt has been expressed in this forum recently about people changing grades and living happily ever after. I can't talk about college systems, but in high school all the grades, attendence, etc. for my high school and several other high schools were kept on a mainframe in a centralized location. There is a system in Michigan called MOIS for vocational data, and on the back of my MOIScript on computer science was the transcript of a terminal session between the attendance people and the computer. The login message gave the phone number of the mainframe. The password was overprinted, but that is useless -- you can learn to read through almost any overprinting. Access to the grading, course scheduling, and attendance programs was by providing a social security number which was echoed and not overprinted. I thus found myself able to do really miraculous things. Being sixth in my high-school class, I had no real motivation to use my knowledge maliciously, and informed the administration. Some safeguards were added (old social security numbers retired, certain social security numbers only giving access to certain programs, restricting access to certain programs to certain accounts), but I'm sure those could have been circumvented with minimal effort -- I was a fairly good systems hack on the operating system, and there were people who could hack rings around me. The operating system was a simple three-tier ring system, and a lot of the features put in for the sake of usability made it very insecure. I send this to give first-hand evidence to those who have been posting doubts that such things happen. ------------------------------ Date: Wed, 16 Oct 85 17:11:36 EDT From: Andy_Mondore%RPI-MTS.Mailnet@MIT-MULTICS.ARPA To: RISKS@sri-csl.arpa Subject: Databases, grades, etc. One of the systems programmers here at RPI made a point about administrators and students sharing the same computer: You really aren't that much more secure when you have administrators and students on separate computers if there is a network or dial-up connection to the administrative computer than you are when administrators and students are on the same computer. If you have network or dial-up connections to an administrative computer, it isn't too difficult for a student with an autodial modem to write a program on a PC that simply tries all possible phone numbers for certain telephone exchanges and record the numbers that produce a carrier tone. Then the student could have another program that tries passwords unless the system disconnects the line after a certain number of unsuccessful tries. The major advantage of separating administrators and students is that it might be more difficult for a student to access an administrative file from a student account assuming the administrative file has the proper protection set. ------------------------------ Date: Mon, 21 Oct 85 0:44:24 EDT From: Sienkiew@louie.udel.EDU To: risks@louie.udel.EDU Subject: Database, Grades, etc... You can create an extremely effective audit trail if you think about it for a while. That just makes the problem more "challenging". Suppose you do your auditing one day and find that there were unauthorized grade changes made for every student in the CS department and 1/2 of them requested (incorrect) printed transcripts. It seems unlikely that everybody independently broke in on the same day. So who do you penalize? How many transcripts have been mailed out already? Suppose no grades were changed but there is a trojan horse waiting to raise the grade only under certain circumstances? My point is that the data doesn't have to stay changed forever. And you can't check the auditing records for every transaction, if you expect to gain by using the computer. You need to do as much as you can, of course, but beware of the false sense of security... Mark. ------------------------------ End of RISKS-FORUM Digest ************************ -------