United States General Accounting Office __________________________________________________________________ GAO Report to the Chairman, Legislation and National Security Subcommittee, Committee on Government Operations, House of Representatives __________________________________________________________________ July 1990 STRATEGIC DEFENSE SYSTEM Stable Design and Adequate Testing Must Precede Decision to Deploy Some of the information in this report--e.g., pictures, charts, and tables--could not be shown in ASCII text format. If you wish to obtain a complete report, call GAO report distribution at 202/275-6241 (7:30 a.m.-5:30 p.m. EST) or write to GAO, P.O. Box 6015, Gaithersburg, MD 20877. _________________________________________________________________ B-239476 July 6, 1990 The Honorable John Conyers, Jr. Chairman, Legislation and National Security Subcommittee Committee on Government Operations House of Representatives Dear Mr. Chairman: This report discusses why the Strategic Defense Initiative Organization will not be able to support currently scheduled full-scale development or deployment decisions on any part of Phase I of the Strategic Defense System. The organization will not be able to support these decisions until it has solidified the system architecture, conducted integrated system-level testing in real time involving actual pieces of hardware and software, and until the Department of Defense adheres to technical and programmatic oversight, commensurate with such a system. As arranged with your office, unless you publicly announce the contents earlier, we plan no further distribution of this report until 30 days from the date of this letter. We will then send copies to other appropriate congressional committees; the Director, Strategic Defense Initiative Organization; the Director, Office of Management and Budget; the Secretary of Defense; and other interested parties. This report was prepared under the direction of Samuel W. Bowlin, Director for Defense and Security Information Systems, who may be reached at (202) 275-4649. Other major contributors are listed in appendix IV. Sincerely yours, Ralph V. Carlone Assistant Comptroller General 1 ____________________________________________________________________ EXECUTIVE SUMMARY ____________________________________________________________________ PURPOSE The Department of Defense plans to spend between $69 and $87 billion to develop and deploy the first of a multiple phase Strategic Defense System designed to shield the United States from a massive Soviet nuclear ballistic missile attack. At the request of the Chairman, Legislation and National Security Subcommittee, House Committee on Government Operations, GAO reviewed the Strategic Defense Initiative Organization's (SDIO's) test and evaluation plans. These plans are to support a 1993 executive decision by the President on deploying Phase I and a 1994-1995 acquisition decision by the Department of Defense to enter full-scale development. Specifically, GAO was asked to determine -- what test and evaluation requirements were established for making development and deployment decisions for the first phase, and -- whether SDIO's approach will meet those requirements and support a decision on deployment. ____________________________________________________________________ BACKGROUND The Strategic Defense System is being designed to destroy thousands of incoming ballistic missiles and warheads. This will require detecting the missiles and warheads, discriminating them from hundreds of thousands of decoys, precisely tracking the missiles and warheads, and then destroying them--all within about 35 minutes or less, and despite the enemy's concerted effort to defeat the system. The Strategic Defense System is, by far, the most complex, technologically challenging system ever attempted. As currently envisioned, a sophisticated communications system will link together hundreds, or possibly thousands, of computer-operated components in space and on the ground. The system will consist of several subsystems (elements), which include surveillance satellites, space- and ground-based weapons, and ground-based subsystems to command and control the satellites and weapons. 2 The Strategic Defense System is being developed in several phases. Phase I will be built using existing technologies; later phases would use more exotic technologies, such as lasers and particle beams. On September 17, 1987, the Secretary of Defense directed that Phase I enter the concept demonstration and validation stage. This stage is important because enough information must be developed to show that the system is feasible before a decision is made to enter full-scale development. Because Phase I cannot be demonstrated outside of an actual ballistic missile attack, much of this information can be generated only through computer simulation and modeling. In 1988, SDIO formally adopted a design (architecture) for Phase I. However, in January 1990 SDIO changed the design dramatically by incorporating a new space-based weapon, called Brilliant Pebbles. The idea behind Brilliant Pebbles is that thousands of small, relatively autonomous weapons would be deployed to intercept Soviet intercontinental ballistic missiles. ____________________________________________________________________ RESULTS IN BRIEF The President is scheduled to make a decision in 1993 on deploying Phase I. His options include deploying, delaying, or cancelling Phase I. Public Law 99-145 states that a Strategic Defense System may not be deployed in whole or in part unless the President certifies to the Congress that the system can fulfill its mission. SDIO may be able to justify a decision to delay or cancel Phase I; however, SDIO will not be able to support an informed decision to deploy it. To do so requires a stable design, sufficient testing and evaluation, and according to SDIO, minimum funding levels. However, it is highly unlikely any of these conditions will be met. The addition of Brilliant Pebbles in January 1990 has reduced, changed, or eliminated the need for some of the space-based elements in the 1988 design. At this point, it is uncertain exactly what Phase I will consist of in terms of elements or what functions they will perform. Additionally, while SDIO initially required real-time integrated system- level testing prior to a presidential deployment decision and a full-scale development decision, SDIO officials now state that such detailed tests will not be conducted prior to either decision. Furthermore, according to SDIO, because of fiscal year 1990 funding cuts, the full-scale development 3 decision has been delayed until 1994-1995, and research efforts have been scaled back, further reducing the amount of information that will be available for a presidential deployment decision. SDIO officials believe they will be able to support a presidential deployment decision with less information than was originally desired, but state that this will be done at increased risk. Phase I has not received the scrutiny and oversight that Defense initially envisioned. The Defense Acquisition Board did not conduct its scheduled 1989 review, nor has it assessed how Brilliant Pebbles affects the design and test and evaluation requirements. The lack of effective agency oversight has contributed to the failure of other automated weapons systems, none of which matches the scale and technological complexity of Phase I. Consequently, any executive decision in 1993 to deploy Phase I would be premature and fraught with high risk. ____________________________________________________________________ PRINCIPAL FINDINGS SDI Design and Test Plans in State of Flux Several problems are hampering SDIO's design and test plans. First, Brilliant Pebbles radically changed the design and sent the program into a state of flux. SDIO has not yet solidified the role of Brilliant Pebbles or what elements will be in the final design. This causes problems because without a stable design detailed Strategic Defense System requirements cannot be determined. Setting requirements is especially important for Phase I where weapon, sensor, and communication systems are interdependent. Changing a requirement for a weapon system could significantly affect sensor and communications systems as well. Further, an unstable design increases the probability that system requirements will not be adequately determined and sufficient testing will not be performed to ensure that the system works. Nevertheless, Defense has requested $265 million for full-scale development of one of the sensor elements, the Boost Surveillance and Tracking System, for fiscal year 1991 even though Brilliant Pebbles may eliminate or reduce the requirements for this system. Second, SDIO does not plan to conduct integrated system-level testing by the scheduled 1993 presidential decision. SDIO-funded studies and test plans have cited the importance of running real-time 4 integrated system-level tests, which combine computer simulation and actual hardware and software prototypes with human intervention. Furthermore, most of the system tests and evaluations so far have been based on the pre- Brilliant Pebbles design. Consequently, much of the testing and analyses may no longer be relevant and will have to be repeated. Also, according to SDIO, its ability to support an informed deployment decision depended on a minimum funding level of $4.6 billion for fiscal year 1990. However, funding was cut by 20 percent. According to SDIO, such a cut will reduce the amount of research performed, delay SDIO's full-scale development decision by 2 years, and reduce the amount of information available to the President. Nevertheless, SDIO officials have stated that they will be able to support a presidential decision on the program even though less information will be available. Original Program Oversight Reduced Because of the complexity, uniqueness, and cost of Phase I, the Defense Acquisition Board initially was to follow a highly structured oversight system of annual reviews looking at cost growth, changing requirements, and test and evaluation results. However, the October 1989 program review scheduled by the Defense Acquisition Board did not occur. According to Defense officials, the review was cancelled because the fiscal year 1990 budget had not yet been finalized and because Defense was reassessing the amount of oversight needed for the Strategic Defense System. Even though a Defense Acquisition Board review is scheduled for June 15, 1990, major changes to the Phase I design and significant reductions to test and evaluation requirements have already occurred without the Board's review and evaluation. ___________________________________________________________________ RECOMMENDATIONS TO THE SECRETARY OF DEFENSE GAO recommends that the Secretary of Defense advise the President to defer a decision to deploy any element of the Phase I system until SDIO has stabilized the design and has demonstrated the effectiveness of the system through integrated system-level tests in real time, using system hardware and software prototypes with human 5 intervention. Further, the Secretary should ensure that required oversight by the Defense Acquisition Board be followed. The Board should more closely monitor system design, development, testing, and evaluation. GAO is making other recommendations in chapter 4. RECOMMENDATION TO THE CONGRESS GAO recommends that the Congress not fund full-scale development for any element of Phase I, including $265 million for the Boost Surveillance and Tracking System in fiscal year 1991, until SDIO has stabilized the design and has demonstrated the effectiveness of the system through integrated system-level tests in real time, using system hardware and software prototypes with human intervention. ____________________________________________________________________ AGENCY COMMENTS As requested by the Chairman's office, GAO did not obtain official agency comments on a draft of this report. However, GAO discussed the information contained in this report with SDIO officials and has incorporated their comments where appropriate. 6 C o n t e n t s --------------- Page ---- EXECUTIVE SUMMARY 2 CHAPTER 1 INTRODUCTION 9 Genesis/Evolution of the SDI Program 9 Objective of a Strategic Defense System 10 Description of Phase I 11 Importance of Concept Demonstration and 12 Validation and Defense Acquisition Oversight Constraints Affecting SDIO's Approach to 13 Concept Demonstration and Validation Objectives, Scope, and Methodology 14 2 UNSTABLE ARCHITECTURE AND TEST PLANS WILL NOT 17 SUPPORT A PRESIDENTIAL DEPLOYMENT DECISION IN 1993 Brilliant Pebbles Sends Phase I and Strategic 17 Defense System into State of Flux SDIO Will Not Perform Integrated System-Level 20 Testing Prior to 1993 SDIO Asserts Funding Cuts Will Reduce Information 22 Needed For an Informed Decision in 1993 3 PROGRAM MANAGEMENT OVERSIGHT 24 HAS BEEN REDUCED Oversight For Phase I More Stringent Than 24 Typical Acquisition 1989 Annual DAB Review Did Not Occur; Decisions 25 Have Been Made Without DAB Review and Approval Congress Cuts Funding for Operational Testing 26 and Evaluation Until Architecture is Solidified 7 4 CONCLUSIONS AND RECOMMENDATIONS 28 Recommendations to the Secretary of 30 Defense Recommendations to the Congress 30 APPENDIXES Appendix I: Department of Defense Major 31 Systems Acquisition Stages Appendix II: Strategic Defense System Elements 33 of Phase I Appendix III: Test Beds Supporting Demonstration 34 and Validation of the Strategic Defense System Appendix IV: Major Contributors to This Report 35 FIGURES (These figures are not included because they could not be shown as ASCII text.) Figure 1.1: Phases of a Ballistic Missile Attack 11 Figure 1.2: Phase I Architecture Incorporating 12 Brilliant Pebbles ABBREVIATIONS ------------- DAB Defense Acquisition Board GAO General Accounting Office IMTEC Information Management And Technology Division SDI Strategic Defense Initiative SDIO Strategic Defense Initiative Organization 8 CHAPTER 1 --------- INTRODUCTION ------------ The Strategic Defense Initiative (SDI) is a program to develop and deploy a Strategic Defense System to shield the United States against a Soviet nuclear ballistic missile attack. Research and development for the SDI program began in 1984, and in 1987 a Strategic Defense System was approved for acquisition. This system is to be developed in multiple phases. Phase I, the subject of this report, is in the concept demonstration and validation stage of the Department of Defense's acquisition process.#1 The Strategic Defense System will be an expensive undertaking. The Strategic Defense Initiative Organization (SDIO) cost estimates range from $69.6 to $87 billion for completion of Phase I development and deployment--with an additional $10.1 billion for Phase I operation and support until deployment is completed. While the cost of follow-on phases is unknown, SDIO estimates that, in addition to the Phase I costs, approximately $26.4 billion will be needed for fiscal years 1990 through 1994 to pursue non-Phase I research and development. GAO has recently reported to the Congress that the above figures are, at best, optimistic.#2 GENESIS/EVOLUTION OF THE SDI PROGRAM ------------------------------------ On March 23, 1983, President Reagan called for a comprehensive scientific research effort to develop a system that would render nuclear ballistic missiles impotent and obsolete. In January 1984, SDI was established as a research and technology development program, and in April 1984, Defense formally chartered the Strategic Defense Initiative Organization as the agency responsible for managing Defense's efforts. After several years of research, SDIO decided in 1987 to develop and deploy the system in phases. SDIO felt that working in phases would allow SDIO to be prepared for an early deployment, if such a decision were made, and to respond to changing threats. Phase I of the system is to be based on currently available technologies, while later phases are to incorporate technologies that are expected to be available in the future, such as neutral particle beams and lasers. Phase I is not intended to be a "total defense", but is being designed to destroy a certain percentage of some intercontinental and sea-launched ballistic missiles (the exact percentages are classified). Creating a full strategic defense 1The Department of Defense's major system acquisition process is supposed to provide a single approach to designing, developing, implementing, and maintaining major weapons systems. (The five stages of the acquisition process are discussed in appendix I.) 2Strategic Defense Initiative: Funding Needs Through Completion of Phase I System (GAO/NSIAD-90-79FS, Jan. 29, 1990). 9 capability will require the deployment of the follow-on phases. Therefore, a critical aspect of the first phase is the capability to evolve and support future phases. Further, the system is not being designed to destroy sea-and air-launched cruise missiles.#3 Other costly systems independent of the SDI program will be needed to defend against these weapons. From the beginning, SDIO has had one overall goal--to conduct a vigorous research and technology program that would provide the basis for an informed decision regarding the deployment of a Strategic Defense System. The current Phase I program is intended to support an executive decision on deployment by the President in 1993 and an acquisition decision on full-scale development by Defense in 1994 or 1995. According to SDIO test and evaluation officials, the President will have a range of options including deploying, delaying, or cancelling Phase I. However, if the President decides to deploy the system, Phase I development will not be consistent with Defense's prudent acquisition policies specifying that deployment decisions be made after full-scale development. OBJECTIVE OF A STRATEGIC DEFENSE SYSTEM --------------------------------------- The Strategic Defense System is based on a layered defense concept; the system is supposed to intercept a missile or a deployed warhead, as it travels toward its target. The basic concept is that separate sensor, weapon, and command and control systems (SDIO refers to these separate systems as elements) would be in space and on the ground. The system would be tied together by a complex communications network and a sophisticated computer- based command and control element. During an attack, the system would have to function in an extremely hostile environment, including nuclear explosions and a concerted effort by the enemy to destroy the Strategic Defense System. The threat the system would be facing, along with the environment in which it would be operating, creates a unique and demanding challenge. First, the space-based elements of the Strategic Defense System would have to be able to detect and begin tracking thousands of missiles almost immediately after launch, and then intercept and destroy some of them. Those missiles that are not destroyed would release warheads, along with decoys intended to confuse the system, forcing the system to track hundreds of thousands of objects. Also at this point, space-based elements of the system would have to distinguish the warheads from the decoys, and intercept and destroy some of the warheads. The system would need to continue discriminating and tracking the surviving warheads so that the ground-based elements could intercept more warheads before they hit the United States. These functions would have to be tightly coordinated and performed in less than 35 minutes--all this with nuclear warheads exploding, anti-satellite weapons attacking the system, and the enemy trying to disrupt 3Cruise missiles are guided missiles that have terrain-seeking radar and fly at moderate speed in low altitude. 10 communications and computer operations. Figure 1.1 describes the phases of a ballistic missile attack. Figure 1.1: Phases of a Ballistic Missile Attack (This figure is not included because it could not be shown as ASCII text.) Source: Adapted from SDI Technology Survivability and Software, Office of Technology Assessment, May 1988. DESCRIPTION OF PHASE I ---------------------- In 1988, a Phase I design, made up of seven elements, was approved by the Defense Acquisition Board. The design consisted of two space-based sensors (Boost Surveillance and Tracking System and Space Surveillance and Tracking System); a space-based weapon (Space-Based Interceptor); two ground-based sensors (Ground Surveillance and Tracking System and Ground-Based Radar); and a ground-based weapon (Ground-Based Interceptor), all managed by a command and control system (Command Center Element). (Appendix II describes each element.) The six surveillance and weapon elements would be highly interdependent and rely heavily on the command and control element, along with a complex communications system. The individual elements would work together as an integrated system to detect, track, discriminate, and destroy ballistic missiles and their warheads. At the heart of the system is a large, distributed, real-time computer software system which, by some estimates, could have 40-100 million lines of code. The complexity and risk of this undertaking cannot be overstated. We have reported repeatedly on Defense systems that were far less complex, contained far fewer lines of code, but have yet to perform as intended.#4 In January 1990, SDIO decided to include a new weapon concept, called Brilliant Pebbles, in Phase I. Brilliant Pebbles involves several thousand individual interceptors orbiting the earth in order to detect and destroy a target by smashing into it at high speed. Brilliant Pebbles is supposed to improve system survivability and reduce costs by dispersing thousands of space- based interceptors that are smaller and more autonomous than the Space-Based Interceptor. The Space-Based Interceptor would house a number of interceptors, but unlike Brilliant Pebbles, it would rely on other satellites for tracking, targeting, and 4Military Space Operations: Operational Problems Continue With the Satellite Control Computer System (GAO/IMTEC-89-56, Aug. 8, 1989); Space Defense: Management and Technical Problems Delay Operations Center Acquisition (GAO/IMTEC-89-18, Apr. 20, 1989); Attack Warning: NORAD's Communications System Segment Replacement Program Should Be Reassessed (GAO/IMTEC-89-1, Nov. 30, 1988); Military Space Operations: Shuttle and Satellite Computer Systems Do Not Meet Performance Objectives (GAO/IMTEC-88-7, Aug. 5, 1988). 11 communications. By making interceptors autonomous, they would no longer need to rely on other satellites to perform these functions. Further, because each interceptor would work independently, the system's survivability would be increased, in principle, because the loss of any one interceptor would not greatly affect the system's overall effectiveness. Figure 1.2 shows Phase I with the six elements and Brilliant Pebbles. Figure 1.2: Phase I Architecture Incorporating Brilliant Pebbles (This figure is not included because it could not be shown as ASCII text.) SDIO funded three Brilliant Pebbles research and development studies during 1989 and 1990. Studies completed by the Defense Science Board and the JASONs--a group of scientists who periodically provide technical support to the Department of Defense--determined that the Brilliant Pebbles concept was technically feasible. Along with these two studies, Defense issued a Space-Based Architecture Study which reviewed the four space-based elements of Phase I--Space Surveillance and Tracking System, Boost Surveillance and Tracking System, the Space-Based Interceptor, and Brilliant Pebbles--and defined and justified a recommended space-based architecture. The Space-Based Architecture Study recommended that research continue on both Brilliant Pebbles and the Space-Based Interceptor and that modified versions of the Boost Surveillance and Tracking System and the Space Surveillance and Tracking System sensors be included. SDIO has decided to eliminate the Space-Based Interceptor from Phase I, claiming that Brilliant Pebbles will cut costs by 20 percent, that is, from $87 to $69.6 billion. However, it is unclear whether the costs for all Phase I requirements have been incorporated into the $69.6 billion. Further, neither figure includes the costs for operating and maintaining the system. IMPORTANCE OF CONCEPT DEMONSTRATION AND VALIDATION -------------------------------------------------- AND DEFENSE ACQUISITION OVERSIGHT --------------------------------- On September 17, 1987, the Secretary of Defense directed that Phase I of the Strategic Defense System enter the concept demonstration and validation stage of Defense's major systems acquisition process. SDIO's basic acquisition strategy is to complete Phase I while researching later phases of a Strategic Defense System. After successfully completing concept demonstration and validation, Phase I could enter full-scale development. According to the Defense Systems Management College, concept demonstration and validation may be the most critical of all acquisition stages. The primary purpose of this stage is to validate the feasibility of Strategic Defense System concepts and technologies, and show that the risk is low enough to enter full- scale development. According to the College, the Secretary of Defense's decision to place a major system in full-scale 12 development is extremely important. During full-scale development, the system is built, tested, and ready for full-rate production. Not only will development consume enormous resources, but major systems, at this point, take on a life of their own and are seldom cancelled. To help Defense gain sufficient design and development information, major system acquisitions are reviewed by the Defense Acquisition Board (DAB). The purpose of the DAB, which is chaired by the Under Secretary of Defense for Acquisition, is to oversee major Defense acquisitions. The DAB reviews each acquisition stage to ensure that every program is ready to proceed into more advanced stages of development or production. Each stage of the acquisition process, as well as each milestone decision, is to be supported by test and evaluation. The purpose of test and evaluation is to help ensure the timely development, production, and fielding of systems that meet users' requirements and perform as intended. The DAB relies on information from test and evaluation to determine whether a system is ready to enter full-scale development. The DAB reviews also ensure that plans for later stages follow sound acquisition management practices. The complexity, cost, and uniqueness of the Strategic Defense System has prompted the need for a more detailed program review process. For example, the DAB developed a guidance document called the Implementor, which provides a framework for additional oversight. Although formal DAB reviews are usually only required at major milestones, the Implementor recommends yearly DAB reviews. Further, SDIO must submit additional documentation to the DAB to ensure program goals are being met. CONSTRAINTS AFFECTING SDIO's APPROACH ------------------------------------- TO CONCEPT DEMONSTRATION AND VALIDATION --------------------------------------- Constraints exist which affect how the Strategic Defense System can be demonstrated. Although the 1972 Anti-Ballistic Missile Treaty permits research, it limits the development, testing, and deployment of different types of strategic defense systems and components. According to SDIO's 1989 report to the Congress on SDI compliance with the treaty, research includes conceptual design and laboratory testing. Development occurs after research but precedes full-scale testing of systems and components designed for actual deployment. The development, testing, and deployment of launchers, interceptors, and radars are restricted. Also, the extent to which the system can demonstrate its ability to interact in a nuclear environment is constrained by limits on nuclear testing. Because the Strategic Defense System cannot be tested in its operational environment, many system capabilities must be demonstrated through computer modeling and simulation. For example, in simulations, software models would mimic the behavior of sensors and weapons and be used in place of the actual elements to evaluate system performance. A ballistic missile attack from launch to impact must also be simulated in software to prompt the element models to respond. Eventually, some of these tests would 13 involve actual prototypes#5 of weapons and sensor hardware and software interacting in the simulation. This type of integrated system-level testing would be used to evaluate the performance of elements within the context of the entire Strategic Defense System. Thus, the ability to simulate the interaction of the systems' hundreds of computers with their many millions of lines of software code becomes of paramount importance in demonstrating whether the system can perform its mission. Early in the SDI program, SDIO recognized the need for integrated system-level testing prior to a full-scale development decision. Defense policies state that models or simulations can support a full-scale development decision and that prototyping is one of the most powerful tools available for determining system feasibility and capability. Consequently, SDIO initially planned to build a simulation framework to conduct tests using software and hardware prototypes. SDIO studies and contractor documents have supported the need for such test capabilities before a full-scale development decision. However, because SDIO believed building such a simulation framework would be very complex, it explored other approaches for demonstrating and validating the Phase I system. SDIO decided instead to use multiple test beds.#6 Under this approach, each test bed would address critical strategic defense issues such as system performance, command and control, communications, and sensing and tracking. The system performance test bed, known as the System Simulator, would be at the core of this test environment. The System Simulator, using computer models of individual elements, would perform end-to-end, system- level tests in which all aspects of Phase I's performance would be evaluated while under a simulated ballistic missile attack. Although the System Simulator, or some other simulation framework, might evolve to include real-time integrated hardware-, software-, and human-in-the-loop capabilities, SDIO has no plans to accomplish this prior to either the presidential decision on deployment or the full-scale development decision. OBJECTIVES, SCOPE, AND METHODOLOGY ---------------------------------- On June 5, 1989, the Chairman, Legislation and National Security Subcommittee, House Committee on Government Operations, requested that we assess the Strategic Defense Initiative Organization's program activities, test and evaluation requirements, and decisions on the development and deployment of Phase I of the Strategic Defense System. Specifically, the objectives of our review were to determine (1) what test and evaluation requirements 5A prototype is an original or model on which a later item is formed or based. A prototype is usually built during the concept demonstration and validation stage and tested prior to the full- scale development decision. 6Test beds are sites, facilities, or activities used for testing models or prototypes. SDIO refers to its network of geographically dispersed test facilities as the National Test Bed. 14 were established for making development and deployment decisions for the first phase of the Strategic Defense System and (2) whether the current program approach will meet those requirements and support a decision on whether to deploy. To address our objectives, we interviewed a wide range of officials involved in the Strategic Defense Initiative. SDIO representatives included: (1) the Phase I System Engineer, as well as other Engineering Office officials, (2) test and evaluation officials, (3) Brilliant Pebbles Program Office officials, and (4) National Test Bed Program Office officials. We discussed technical issues with members of the Phase One Engineering Team, which acts as a consultant to SDIO. Team members are experts in ballistic missile defense and include physicists, software and electrical engineers, and simulation analysts. We also met with officials from Martin Marietta, the prime contractor for the National Test Bed, and officials from General Electric Company, the system engineer and integration contractor for the Strategic Defense System. General Electric has the responsibility for defining Phase I demonstration and validation requirements, including those for test and evaluation. We met with Army and Air Force officials involved in major SDI research and development as well as Defense Acquisition Board officials. Additionally, we consulted with ballistic missile defense experts at Sandia and Los Alamos National Laboratories. Our approach was to obtain a thorough understanding of the underlying engineering concepts for the Strategic Defense System by (1) obtaining and analyzing key system design and requirements documents, (2) interviewing and consulting with experts from the ballistic missile defense community, and (3) visiting and assessing research and development sites. We reviewed pertinent management, technical, and contract documents provided by SDIO, SDIO contractors, the Air Force, the Army, Los Alamos National Laboratory, and the Defense Science Board. We analyzed and compared system documents such as the 1988 System Description Document, the 1990 System Description Document, the Brilliant Pebbles System Description Document, and the Space-Based Architecture Study. Furthermore, we analyzed and compared test and evaluation documents including the 1987 and 1989 Test and Evaluation Master Plan, the Phase I Integrated Test Plan, the System Simulator requirements document, the technical specifications and software for the current System Simulator, the Brilliant Pebbles Integrated Test Plan, and the Brilliant Pebbles Test and Evaluation Master Plan. Through our analyses we were able to identify the chronology of events that led to SDIO's current test and evaluation approach in support of decisions on deployment and full-scale development. We did not, however, validate test and evaluation results. Our work was conducted at SDIO Headquarters, Washington, D.C.; the National Test Bed Joint Program Office, Falcon Air Force Base, Colorado Springs, Colorado; Electronic Systems Division, Boston, Massachusetts; the Army Strategic Defense Command, Huntsville, Alabama; Sandia National Laboratory, Albuquerque, New Mexico; Los Alamos National Laboratory, Los Alamos, New Mexico; the Phase One 15 Engineering Office, Crystal City, Virginia; the Defense Acquisition Board, Washington, D.C.; General Electric Corporation, Blue Bell, Pennsylvania; and Martin Marietta Corporation, Colorado Springs, Colorado. We discussed the facts in this report with SDIO officials and have incorporated their comments where appropriate. However, in accordance with the requester's wishes, we did not obtain official agency comments on a draft of the report. We conducted our work from June 1989 through May 1990, in accordance with generally accepted government auditing standards. 16 CHAPTER 2 --------- UNSTABLE ARCHITECTURE AND TEST PLANS WILL NOT --------------------------------------------- SUPPORT A PRESIDENTIAL DEPLOYMENT DECISION IN 1993 -------------------------------------------------- The