Safety Assesment Before Investment Decision

帅哥

FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 1 Chapter 4: Safety Assessments Before Investment Decision 4.0 SAFETY ASSESSMENTS BEFORE INVESTMENT DECISION...............................................2 4.1 OPERATIONAL SAFETY ASSESSMENT ...................................................................................3 4.2 COMPARATIVE SAFETY ASSESSMENT (CSA) ....................................................................10 FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 2 4.0 Safety Assessments Before Investment Decision Before the investment decision at JRC 2, there are two phases of the acquisition life cycle: Mission Analysis and Investment Analysis. The Pre-Investment phase of a program encompasses the Mission Analysis and Investment Analysis phases of the Acquisition cycle illustrated in Figure 4-1. System safety’s purpose during these phases is twofold. The first purpose is to develop early safety requirements that form the foundation of the safety and system engineering efforts. The second purpose is to provide objective safety data to the management activity when making decisions. The early assessment of alternatives saves time and money, and permits the “decision makers” to make informed, data driven decisions when considering alternatives. This section describes the System Safety assessments typically performed prior to the decision to approve a Mission Need at JRC-1, and prior to the decision to go forward with the program at JRC-2. The pre-investment safety assessments are: (1) Operational Safety Assessment (OSA) and (2) Comparative Safety Assessment (CSA). System Safety Products in the AMS Life Cycle Hazard Tracking & Incident Investig ation Track Medium and High Risks Closed Loop/Risk Accep tance Capture & Analyze Incidents Identify high risk trends for further detailed inv estigation Operating and Support Hazard Analysis (O&SHA) - Operating hazards (focus on the human errors/factors details - S upport and Maintenance Hazards System Hazard Analysis (SHA) - Looks at interfaces and env ironment (operating and amb ient) - NAS System Level Subsystem Hazard Analysis (SSHA) - NOT components (next level below System - Focus on faults and hazards at SS level - Detailed - A few safety requirements Comparative Safety Assessment (CSA)/Preliminary Hazard Analysis (PHA) - Top - down, focus on known system mission and approaches and changes at NAS system level - Preliminary in nature - Core Safety Requirements OSA - - System Level - - Preliminary (some assumptions) - S ome Safety Req uirements INTEGRATED PRODUCT DEVELOPMENT S YSTEM may fall out Figure 4-1: Safety Products in AMS Life Cycle An Operational Safety Assessment (OSA) has been designed to provide a disciplined, and internationally developed (RTCA SC189) method of objectively assessing the safety requirements of aerospace systems. In the FAA, the OSA is used to evaluate Communication, Navigation, Surveillance (CNS) and Air Traffic Management (ATM) systems. The OSA identifies and provides an assessment of the hazards in a system, FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 3 defines safety requirements, and builds a foundation for follow-on institutional safety analyses related to Investment Analysis, Solution Implementation, In-Service Management, and Service Life Extension. The OSA is composed of two fundamental elements: (1) the Operational Services & Environment Description (OSED), and (2) an Operational Hazard Assessment (OHA). The OSED is a description of the system physical and functional characteristics, the environment’s physical and functional characteristics, air traffic services, and operational procedures. This description includes both the ground and air elements of the system to be analyzed. The OHA is a qualitative safety assessment of the operational hazards associated with the OSED. Each hazard is classified according to its potential severity. Each classified hazard is then mapped to a safety objective based on probability of occurrence. In general, as severity increases, the safety objective is to decrease probability of occurrence. The information contained in the OSA supports the early definition of system level requirements. It is not a risk assessment in a classical sense. Instead, the OSA’s function is to determine the system’s requirements early in the life cycle. The early identification and documentation of these requirements may improve system integration, lower developmental costs, and increase system performance and probability of program success. While the OSA itself is not a risk assessment, it does support further safety risk assessments that are required by FAA Order 8040.4. The follow-on safety assessments may build on the OSA’s OSED and OHA, by using the hazard list, system descriptions, and severity codes identified in the OSA. The OSA also provides an essential input into CSA safety assessments that support trade studies and decision making in the operational and acquisition processes. The CSA is a safety assessment performed by system safety to assess the hazards and relative risks associated with alternatives in a change proposal. The alternatives can be design changes, procedure changes, or program changes. It is useful in trade studies and in decision-making activities where one or more options are being compared in a system or alternative evaluation. This type of risk assessment can be used by management to compare and rank risk reduction alternatives. More details on how to perform a CSA are included in section 4.2. 4.1 Operational Safety Assessment The OSA is intended to provide system level safety requirements assessment of aerospace CNS/ATM systems. As described above it is composed of two elements: (1) The Operational Environment Definition (OSED) and (2) the Operational Hazard Assessment (OHA). The OSA is based on an RTCA/SC-189 framework. 4.1.1 Operational Environment Definition (OED) The OED is basically a system description that may include all the elements of the 5M model. See chapter 3 for instructions on developing a system description. 4.1.2 OSA Tasks The steps within this task are: · Define the boundaries of the system under consideration. Determine, separate, and document what elements of the system you will describe/analyze from those that you will not FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 4 describe/analyze. The result of this process is a model of the system under analysis that will be used to analyze hazards. · Using models such as those described in chapter 3, describe the system physical and functional characteristics, the environment physical and functional characteristics, air traffic services, human elements (e.g. pilots and controllers, etc.) and operational procedures. · From this description, determine and list the system functions. For example, the primary function of a precision navigation system is to provide CSA and flight crews with vertical and horizontal guidance to the desired landing area. These functions could be split if desired into vertical and horizontal guidance. Supporting functions would be those functions that provide the system the capability to perform the primary function. For instance a supporting function of the precision navigation system would be transmission of the RF energy for horizontal guidance. It is up to the system engineering team to determine how to group these functions and to what level to take the analysis. Detailed analyses would go into the lower level functions. Typically the OSA functional analysis is limited to the top-level functions. See FAA System Engineering Manual for more detailed guidance on functional analysis. 4.1.3 Operational Hazard Assessment The Operational Hazard Assessment (OHA) is the second part of the OSA. The OHA is a qualitative assessment of the hazards associated with the system described in the OSED. Determining functions and hazards Once the system has been bounded, described, and the functions determined in the OSED, the analyst is ready to determine the hazards associated with the system. For these types of assessments the best method is to assess scenarios containing a set of hazardous conditions. Therefore, the following definition can be used to define the hazards in a Preliminary Hazard List (PHL): Hazard The potential for harm. Unsafe acts or unsafe conditions that could result in an accident. (A hazard is not an accident). Hazard or hazardous condition. Anything, real or potential, that could make possible, or contribute to making possible, an accident. Hazard. A condition that is prerequisite to an accident Since the work has already been done in defining the system operational environment, it is often best to relate the functions of the system to hazards. For example, in analyzing the NAS, one would find the following functions of the NAS (listed in Table 4.1-1). These functions are then translated into hazards that would be included in the preliminary hazard list. For many of the listed hazards other conditions must be present before an accident could occur. These are detailed in the detailed description of the risk assessment. The purpose here is to develop a concise, clear, and understandable PHL. FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 5 Table 4-1: Examples of NAS System Functions and Their Associated Hazards NAS System function NAS System hazard Provide air – ground voice communications. Loss of air – ground voice communication. Provide CSA precision approach instrument guidance to runways. Loss of precision instrument guidance to the runway. Provide En Route Flight Advisories of severe weather. Lack EFAS warning of severe weather in flight path to CSA flight crew. In addition to the functional analysis, the following tools can be used to identify the foreseeable hazards to the system operation. These tools are listed in Table 4-2. Determining Severity of Consequence The severity of each hazard is determined by the worst credible outcome, or effect of the hazard on the CSA or system. This is done in accordance with MIL-STD-882 and FAR/AMJ 25.1309. Both documents state that the severity should consider all relevant stages of operation/flight and worst case conditions. See the risk determination Table 3-2 to define the severity levels of a hazard. Table 4-2: Safety Analysis Tools OPERATIONS ANALYSIS Purpose: To understand the flow of events. Method: List events in sequence. May use time checks. PRELIMINARY HAZARD ANALYSIS (PHA) Purpose: To get a quick hazard survey of all phases of an operation. In low hazard situations the PHA may be the final Hazard ID tool. Method: Tie it to the operations analysis. Quickly assess hazards using scenario thinking, brainstorming, experts, accident data, and regulations. Considers all phases of operations and provides early identification of highest risk areas. Helps prioritize area for further analysis. “WHAT IF” TOOL Purpose: To capture the input of operational personnel in a brainstorming-like environment. Method: Choose an area (not the entire operation), get a group and generate as many “what ifs” as possible. SCENARIO PROCESS TOOL Purpose: To use imagination and visualizations to capture unusual hazards. Method: Using the operations analysis as a guide, visualize the flow of events. LOGIC DIAGRAM Purpose: To add detail and rigor to the process through the use of graphic trees. Method: Three types of diagrams- positive, negative, and risk event. FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 6 CHANGE ANALYSIS Purpose: To detect the hazard implications of both planned and unplanned change. Method: Compare the current situation to a previous situation. CAUSE & EFFECT TOOL -- CHANGE ANALYSIS Purpose: To add depth and increased structure to the Hazard ID process through the use of graphic trees. Method: Draw the basic cause and effect diagram on a worksheet. Use a team knowledgeable of the operation to develop causal factors for each branch. Can be used as a positive or negative diagram. Purpose: To detect the hazard implications of both planned and unplanned change. Method: Compare the current situation to a previous situation. CAUSE & EFFECT TOOL Purpose: To add depth and increased structure to the Hazard ID process through the use of graphic trees. Method: Draw the basic cause and effect diagram on a worksheet. Use a team knowledgeable of the operation to develop causal factors for each branch. Can be used as a positive or negative diagram. OHA Tasks The tasks to be accomplished in this phase are: · From the function list (or tools listed in Table 4-2) develop the list of hazards potentially existing in the system under study · Determine the potential severity of each hazard in the hazard list by referring to the risk determination section of Chapter 3. 4.1.4 Allocation of Safety Objectives and Requirements (ASOR) The Allocation of Safety Objectives and Requirements (ASOR) is the process of using hazard severity to determine the objectives and requirements of the system. There are two levels of requirements in this process: (1) objectives (or goals) and (2) requirements (or minimum levels of acceptable performance). The purpose of the ASOR is to establish requirements that ensure that the probability of a hazard leading to an accident has an inverse relationship to the severity of occurrence. This inverse relationship is called the Target Level of Safety (TLS). For example, a “hazardous” or severity 2 hazard would have a requirement (shown by arrows in Figure 4-1) to show by analysis or test to have a probability of occurrence of Extremely Remote or less than one in one-million operating hours for the fleet or system. The objective or (desired probability) in this case would be Extremely Improbable or one occurrence in one billion per operating hour for the fleet or system. See Figure 4-2 for the steps in this process. Once the TLS is determined for each hazard, requirements can be written to ensure that the appropriate hazard controls are established as system requirements. FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 7 Figure 4-2: Target Level of Safety Determination 4.1.5 Identification of High Level Hazard controls The next step is to determine the hazard controls. Controls are measures, design features, warnings, and procedures that mitigate or eliminate risk. They either reduce the severity or probability of a risk. System Safety uses an order of precedence when selecting controls to reduce risk (MIL-STD-882, 1984). This order of precedence as discussed in Section 3.6, and Table 3.6-1 Clearly risk reduction by design is the preferred method of mitigation. But even if the risk is reduced, the term “reduction” still implies the existence of residual risk, which is the risk left over after the controls are applied. For example, residual risk can be controlled in a manner described in Table 4-3. This table describes the NAS System Function, NAS System Hazard, and NAS System Control. 1. Determine potential severity of each hazard in the OHA. 2. Map severity to this chart to determine probability requirement (minimum) and objective (desired) Target Level of Safety (TLS) 3. Allocate the safety objectives and requirements (ASOR) from the TLS to air and/or ground elements Steps Hazard Classification Likelihood Sever ity Probable A Major 3 Catastrophic 1 Hazardous 2 Minor 4 No Safety Effect 5 Remote B Extremely Remote C Extremely Improbable D High Risk Medium Risk Low Risk FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 8 Table 4-3: Development of Controls for Hazards in the NAS NAS System function NAS System hazard NAS System Controls Provide air - ground communications. Loss of air – ground communication. Multiple communication channels. Multiple radios. Procedures for loss of communication. Phase dependent: communication is not always critical. Provide CSA precision approach instrument guidance to runways. Loss of precision instrument guidance to the runway. Reliability. Alternate approaches available. Procedures for alternate airport selection. Fuel reserve procedures. System detection and alert to CSA. Phase and condition (IMC vs. VMC) dependent. Provide En Route Flight Advisories of severe weather. Lack EFAS warning of severe weather to CSA flight crew. Early detection systems (satellite) for severe weather. Multiple dissemination means. Procedures (condition dependent) require alternate airports. Fuel reserve procedures. As the engineer performs the assessment, controls that do not yet exist can be identified and listed. These controls are included in the requirements of the OSA. This is done by turning the controls into measurable and testable requirements or “shall” statements. A critical function of System Engineering is the determination and allocation of requirements early in the concept and definition phase. System Safety’s function in this process is to develop safety-related requirements early in the design to facilitate System Engineering. A primary source of safety requirements is the OSA. The controls identified, both existing and recommended, should be translated into a set of system level requirements. For example, Table 4-4 lists the same hazards and controls that were examined in Table 4-3. The requirements are examples only and are meant for illustration. FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 9 Table 4-4: Examples of Controls and Requirements NAS System Function NAS System Hazard NAS System Controls NAS System Requirements Provide air to ground communicat- ions and control. Loss of air to ground communication and control. Multiple communication channels. Multiple radios. Procedures for loss of communication. Phase dependent: communication is not always critical. The NAS system shall provide for multiple communication modes in the enroute structure, at least 2 channels in each region being in the VHF frequency spectrum, and one available through the satellite communication system. The total Mean Time Between Failure (MTBF) of these systems may not be less than X hours. Provide CSA precision approach instrument guidance to runways. Loss of precision instrument guidance to the runway. Reliability. Alternate approaches available. Procedures for alternate airport selection. Fuel reserve procedures. System detection and alert to CSA. Phase and condition (IMC vs. VMC) dependent. The NAS shall provide at least two backup non-precision approaches at each airport with a precision approach capability. The NAS procedures shall require part 121 operators to select an alternate destination if the forecast weather at the planned destination is less than 500’ and 1 mile over the destinations weather planning minimums within one hour of the planned arrival. Provide Enroute Flight Advisories of severe weather. Lack EFAS warning of severe weather to CSA flight crew. Early detection systems (satellite) for severe weather. Multiple dissemination means. Procedures (condition dependent) require alternate airports. Fuel reserve procedures. The NAS shall detect icing conditions greater than moderate accretion when it actually exists in any area of 10 miles square and at least 1000’ thick for greater than 15 minutes duration. Tasks in the ASOR phase Determine existing and recommended hazard controls for each hazard. Develop requirements based on the TLS and controls. · Allocate the requirements so that both ground CNS/ATM and airborne systems share the controls. FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 10 4.2 COMPARATIVE SAFETY ASSESSMENT (CSA) Comparative Safety Assessments (CSAs) are performed to assist management in the process of decision making. The CSA is a risk assessment, in that it defines both severity and likelihood in terms of the current risk of the system. Whereas an OSA defines the target level of safety, a risk assessment provides an estimation of the risk associated with the identified hazards. The first step within the CSA process involves describing the system under study in terms of the 5M model (chapter 3). Since most decisions are a selection of alternatives, each alternative must be described in sufficient detail to ensure the audience can understand the hazards and risks evaluated. Many times one of the alternatives will be “no change”, or retaining the baseline system. A preliminary hazard list (PHL) is developed and then each hazard’s risk is assessed in the context of the alternatives. After this is done, requirements and recommendations can be made based on the data in the CSA. A CSA should be written so that the decision-maker can clearly distinguish the relative safety merit of each alternative. An example (with instructions) of a CSA is included in Appendix B. 4.2.1 Principles of Comparative Safety Assessments In general, CSA should: Be objective Be unbiased Include all relevant data Use assumptions only if specific information is not available. If assumptions are made they should be conservative and clearly identified. Assumptions should be made in such a manner that they do not adversely affect the safety of the system. Define risk in terms of severity and likelihood in accordance with chapter 3, paragraph 3.4. Severity is independent of likelihood in that it can and should be defined without considering likelihood of occurrence. Likelihood is dependent on severity. The definition of likelihood should be made on how often an accident can be expected to occur, not how often the hazard occurs. Compare the results of the risk assessment of each hazard for each alternative considered in order to rank the alternatives for decision making purposes. Assess the safety risk reduction or other benefits associated with implementation of and compliance with an alternative under consideration. Assess risk in accordance with the risk determination defined in Tables 3-2 and 3-3. 4.2.2 Steps in performing a CSA Define the system under study in terms of the 5m model described in chapter 3 for the baseline system and all alternatives. Perform a functional analysis in accordance with the FAA System Engineering handbook. This analysis will result in a set of hierarchical functions that the system performs. From the functions and system description, develop a preliminary hazard list as described earlier in this chapter. List these PHL hazard conditions in the form contained in Appendix B Evaluate each hazard – alternative combination for severity using the definitions contained in chapter 3. This must be done in accordance with the principles contained in this manual, which require evaluation of the hazard severity in the context of the worst credible conditions. FAA System Safety Handbook, Chapter 4: Pre-Investment Decision Safety Assessments December 30, 2000 4 - 11 Evaluate the likelihood of occurrence of the hazard conditions resulting in an accident at the level of severity indicated in (4) above. These definitions can be found in chapter 3, Table 7 of this guidebook. This means that the likelihood selected is the probability of an accident happening in the conditions described in (4), and not the probability of just the hazard occurring. Document the assumptions and justification for how severity and likelihood for each hazard condition was determined.