Air Traffic Organization Safety Evaluations and Audits

帅哥

Distribution: A-FAT-O; ZAT-464 Initiated By: AJS-0 1. Purpose of this Order. This order provides direction, processes, and procedures for conducting evaluations and audits of Air Traffic Organization (ATO) facilities and services. This order provides guidance for evaluating and auditing compliance with Federal Aviation Administration (FAA) directives and procedures at: FAA ATO facilities; FAA Contract Towers (FCT); Automated Flight Service Stations (AFSS); Flight Service Stations (FSS); FAA Contract Flight Service Stations (FCFSS); Non-Federal Airport Control Towers (NFCT); Systems Operations at the Air Traffic Control System Command Center (ATCSCC) and all field traffic management units; and when requested by the military ATC facilities. This order does not apply to occupational safety, health, or environmental evaluations and audits. 2. Audience. All ATO Personnel, Mike Monroney Aeronautical Center, the William J. Hughes Technical Center, FCTs and NFCTs. 3. Where can I find this Order. You can find this Order on the Directives Management System website: https://employees.faa.gov/tools_resources/orders_notices/ 4. Explanation of Changes. a. To meet current business structure and needed clarification. b. To include Technical Operations in the Safety Evaluations and Audits program. c. To change Mitigation Plan update interval from 30 to 45 days. d. To change the facility internal evaluation completion date from August 1 to September 30. e. To change audits of ARTCCs, ATCTs, ATCs and FCTs to once every three years. f. To change the Management Effectiveness rating identifier from “M” to “E” for the Flight Service option. 5. Effective date of this Order. This order is effective June 24, 2008. 6. What this Order Cancels. Order 7010.1S, Air Traffic Evaluations, dated October 1, 2005, is cancelled. June 24, 2008 Air Traffic Organization Policy SUBJ: Air Traffic Organization Safety Evaluations And Audits ORDER JO 7010.1T 7010.1T 06/24/08 7. Applicability. The following processes and procedures apply to all FAA ATO facilities, and FCTs. AFSSs, FSSs and FCFSSs are outlined separately in Appendix 1. This order excludes Joint Surveillance System (JSS) facilities. The process for NFCTs is outlined in Appendix 2. 8. References. a. The process for Technical Operations NAS Technical Evaluation Program is contained in FAA Order 6040.6. b. The process for military ATC facilities is contained in FAA Order 7610.4. c. Joint Surveillance System facilities are defined in FAA Order 6430.2. 9. Definitions. a. Safety Evaluations, part of the ATO’s Safety Assurance directorate, is the headquarters-based directorate, including all remote offices and any individual assigned to the organization, responsible for evaluation and auditing compliance of FAA directives and procedures of all ATO facilities. b. Facility Manager for purposes of this order refers to the Air Traffic Facility Manager, the SSC Manager or the Traffic Management Officer. c. Lead Evaluator is the Safety Evaluations specialist in charge of all aspects of an audit or assessment. d. Checklist is a compilation of items related to the safe and efficient operation of an ATO facility. It is used as minimum guidance in preparing for and conducting facility evaluations and audits. Appended items can be added to Facility Safety Assessment System (FSAS). e. Evaluation Types. (1) Internal Facility Evaluation. This is an evaluation conducted by a designated representative(s) of the Facility Manager. (2) Military Facility Evaluations. This is an evaluation conducted in accordance with FAA Order 7610.4, Special Military Operations, for military approach controls, military ATCTs at joint use airports, and Ground Control Approach Unit if associated. f. Audit. An audit is an independent method of assessing an organization compliance with FAA directives and procedures. It is administered by Safety Evaluations through random sampling methods such as, but not limited to, direct operational observation, discussions with facility personnel, review of voice or radar data, equipment key parameters, certification parameters, and examination of other documentation. g. Program Assessment. A program assessment is a Safety Evaluations review of an organization’s safety programs or initiatives. Programs and initiatives include, but are not limited to, 2 06/24/08 7010.1T Service Area Quality Assurance, Air Traffic Facility Quality Assurance, Runway Incursion Prevention Plans, Operational Error Reduction Programs, Equipment Availability Programs, and Contractor Quality Assurance programs for FCTs. h. Facility Safety Assessment System (FSAS). FSAS is a national database that contains information related to the Safety Evaluations process. Information includes evaluation checklists, reports, facility information, tracking information, response data, and other statistical information available on the FSAS website. Information contained in reports, mitigation plans, status reports, and closure is submitted through this database system. i. Ratings. A ranking given to each checklist item according to an ATC facility’s compliance with standards as follows: 1) “M” Meets Requirements. This rating is assigned to a checklist item that is in compliance with national, Service Area and local requirements. 2) “N” Needs Improvement. This rating is assigned to a checklist item that complies with some national, Service Area, and local requirements, though several instances of variance from the standard are observed. This rating requires comment, a mitigation plan, and follow-up. 3) “D” Does Not Meet Requirements. This rating is assigned to a checklist item that does not comply with national, Service Area and local requirements. This rating requires comment, a mitigation plan, and follow-up. 4) “A” Requires Immediate Action. This rating is assigned to a checklist item with materiality to safety that requires immediate action. Immediate is defined as commencing on the same day that the item rating is assigned or received. This rating requires notification of the Service Area and Service Unit to obtain mitigation plan concurrence. 5) “N/A” Not Applicable. This rating item is assigned to any checklist item that does not apply to the facility being evaluated. j. Percentage of Compliance. A numerical rating assigned to each checklist item during an evaluation or audit. Example: (0 to100%) k. Item Resolution Process. This is the required method to resolve items rated “N,” “D,” and “A” to the state of “M.” j. Mitigation Plan. The strategy prepared by the Facility Manager to delineate corrective measures for items rated as “N,” “D,” or “A.” At a minimum, this plan includes a clear statement of the corrective measures, the party responsible for the corrective measures, and a specific date and time by which the corrective measures will be completed. This plan requires Service Area or Manager of System Efficiency concurrence for all items. 1) Service Unit concurrence for “A” items and allows for District Office Manager or DTO recommendations on all items. 3 7010.1T 06/24/08 2) Status Report. This report is recorded in FSAS every 45 calendar days to communicate the status of an item rated less than “M” until the item is raised to the “M” level. The report includes progress in resolving each item, including actions, dates, and results. The report will indicate when an item is raised to the “M” level. Recipients of the report include the Service Area, Service Unit, Safety Evaluations, and Office of Primary Interest. However, the report is accessible via the FAA intranet. 3) Closure. This is the final step that must be taken to resolve “N,” “D,” and “A” items. To close an item, the rating must be updated to “M” and the Facility Manager must then approve the rating and update FSAS. 10. Responsibilities. a. Facility Managers must: (1) Complete a minimum of one internal facility evaluation annually, no later than September 30th, unless an alternate completion date is negotiated with the Service Area and approved by ATO-S Evaluations. (2) If an “A” item is identified during an evaluation, initiate immediate (i.e., same day) discussion with the District Manager, Service Area, and Service Unit to agree upon a mitigation plan. The parties will confer and agree upon the plan the same day the item receives an “A” rating. The plan must also be recorded in FSAS upon evaluation or audit approval. (3) For items not rated as “Meets Requirements,” report these in FSAS. Include mitigation plans to correct these items for the District Manager or DTO and Service Area to review. (4) Certify that the facility evaluation is complete and accurate in FSAS as identified. (5) In the case of an audit, start mitigation plans and coordination responsibilities in FSAS for audit items rated “A” on the same day of receipt of that rating. (6) In the case of an audit, submit mitigation plans in response to audit items rated “N” or “D” within 15 calendar days of receipt of the audit report in FSAS. (7) Upon Service Area concurrence of mitigation plans, begin implementation of mitigation plans and provide status reports every 45 calendar days for “N”, “D” and 15 calendar days for “A” items until the items are closed in FSAS. b. District Managers must: (1) Provide oversight and assist facility personnel, as needed, in the evaluation process. (2) Immediately (i.e., same day) confer with the facility manager, Service Area and Service Unit to agree on facility mitigation plans for items rated “A.” 4 06/24/08 7010.1T (3) May respond in FSAS to requests for recommendations on facilities’ mitigation plans for items rated “N” and “D.” (4) Facilitate timely closure of mitigation plans. g. Service Area Director(s) must: (1) Provide oversight and assist the Districts and facilities, as needed, in the evaluation process. (2) Immediately (i.e., same day) confer with the Facility Manager, District Manager and Service Unit to reach consensus on facility mitigation plans for items rated “A.” (3) Prepare responses in FSAS indicating concurrence/non-concurrence with the facilities’ mitigation plans for items rated “N” and “D” within 15 calendar days of receipt. (4) Facilitate timely closure of mitigation plans. d. Service Unit Quality Assurance must: (1) Provide feedback and input to Safety Evaluations program managers about checklist items. (2) Verify items on the checklist are complete and accurate. (3) Provide interpretations on directives pertaining to that Unit’s Office of Primary Interest. (4) Participate in facility audits and operational assessments. (5) Support and concur with Service Area and ATO facilities, for mitigation plans on “A” items. e. Safety Evaluations must: (1) Provide appropriate access to FSAS for air traffic organizations, facilities and support staff. The Vice President of Safety Services will determine access for FSAS by Safety Oversight (AOV) and other non-ATO organizations. (2) Conduct audits and program assessments in accordance with this order. (3) Submit reports in FSAS within 10 calendar days of completing audits. (4) Establish requirements; provide guidance, and review evaluations and audits to ensure that the process is data-driven, effective, and responsive to safety trends. (5) When an interpretation of a checklist item is required, consult the appropriate Service Units Office of Primary Interest for a decision. 5 7010.1T 06/24/08 (6) Provide status reports, at a minimum of every 6 months, of identified issues to Service Units and Offices of Primary Interest, as applicable. (7) Revise FSAS capabilities, as needed. (8) Measure, monitor and report trends ATC system compliance with established policies, procedure and requirements using the evaluation and audit processes. (9) Identify leading indicators of Operational Errors and Deviations (OE/OD), Near Mid-Air Collisions, Controlled Flight into Terrain and other safety hazards. (10) Identify leading indicators of System Availability, Reliability, and other safety hazards. (11) Identify leading indicators of Pilot Deviations and Vehicle Pedestrian Deviations (PD/VPD) that result in either a runway incursion or surface incident and other safety hazards. 11. Evaluation, Audit and Assessment Procedures. a. General. Evaluations and Audits must be conducted using, but not limited to, observation, examination of certification and key parameters, position monitoring, voice and data reviews, documentation review and examinations, interviews, and discussions. b. Internal Facility Evaluations. The Facility Manager or his/her designee must: (1) Initiate an internal facility evaluation and enlist the assistance of the Service Area. (The Service Area may also request that the Facility Manager conduct an internal evaluation). a) The evaluation is not expected to be completed as a single activity. Each quarter, at a minimum, 25% of a facilities checklist items must be completed. b) The evaluation team may be composed of any members the Facility Manager deems appropriate; the team member selection must be in accordance with all applicable national collective bargaining agreements. c) The Regional Runway Safety Program Manager may be invited to observe the evaluation of a facility located within their regional area of responsibility. (2) Ensure each checklist item is assessed and assigned a rating and percentage of compliance as described in paragraph 8i and 8j. a) For items on the checklist that cannot be verified during the course of normal operations (e.g., a bomb threat), evaluators will use every means available to evaluate those items, including, but not limited to, interviews, discussions, simulations (e.g., table-top exercises), and documentation examination. 6 06/24/08 7010.1T b) An evaluation is complete when all items have been rated, a mitigation plan has been submitted, and the report is recorded in FSAS no later than September 30 th of each year. (3) Prepare a report in FSAS that includes mitigation plans for items rated “N,” “D,” and “A.” (4) The Facility Manager must certify in FSAS that the evaluation report is complete and accurate to the best of his/her knowledge. The certification will cause FSAS to generate reports to all appropriate parties. This must be done within 15 days upon evaluation completion. c. Audits. Safety Evaluations must: (1) Determine facilities and programs to be audited. (a) Solicit input from the Service Areas and Lines of Business to establish audit priorities. (b) Use objective criteria from sources, including, but not limited to, OE/D statistics, Air Traffic counts, open evaluation items, Target lists, NASPAS, NASTEP, and length of time since last ATO-S audit. (c) For Terminal, En Route and Oceanic, and Traffic Management Units, perform an on-site audit at all applicable ARTCCs, TRACONs, ATCTs, and FCTs once every three years. (d) For Technical Operations NASTEP, each Service area and Headquarters program office will be audited. Safety Evaluations will conduct a minimum of 6 facility audits per Fiscal Year (FY). (2) Prepare for an audit by reviewing relevant facility operations or program information. (3) Notify the appropriate Service Unit prior to conducting an audit. (4) Conduct an in-briefing to the Facility Manager or his/her designee. The briefing will include an introduction of team members and audit expectations. (5) Assign ratings and percent compliant as described in paragraphs 8h and 8i, to applicable items using any of the methods listed in paragraph 8f. An audit team may elect to assess some or all checklist items. (6) Brief the Facility Manager or his/her designee in person or via a telephone conference within 5 calendar days of completing the audit. (7) Complete the audit report in FSAS within 10 calendar days of the audit. d. Program Assessments. Safety Evaluations must conduct program assessments at each Service Area and Contractor Quality Assurance program once every 2 years, at a minimum. 7 7010.1T 06/24/08 12. Evaluation and Audit Reports. a. Internal Facility Evaluation Reports. A report is automatically generated in FSAS upon completion and certification by the Facility Manager of their annual facility evaluation. Certification must occur no later than October 1. The report must show all items rated and approved and include associated problem statements and mitigation plans for all items rated “N,” “D,” and “A”. b. Audit Reports. The Safety Lead Evaluator must submit a report in FSAS within 10 calendar days of audit completion. The report must list all items rated and include associated problem statements for all items rated “N,” “D,” and “A”. 13. Item Resolution. a. Mitigation Plans. The Facility Manager must submit mitigation plans to the Service Area. The Service Area must provide responses in FSAS to the facility within 15 days of receipt of the plans. b. Status Reports. The Facility Manager or his/her designee must update the status of all open items from evaluations or audits a minimum of every 45 calendar days for “N” or “D” rated items and every 15 days for any “A” rated items in FSAS. A status report will become due every 45 calendar days and this cycle will be repeated until the item is raised to the “M” rating level. At that time, the Facility Manager will certify the item is closed. c. Closure. Once a facility mitigation plan has been implemented and the Facility Manager determines the item meets requirements, the Facility Manager will update the status in FSAS with an “M” rating and close the item. 14. Distribution. FAA ATO facilities; FAA Contract Towers (FCT); Automated Flight Service Stations (AFSS); Flight Service Stations (FSS); FAA Contract Flight Service Stations (FCFSS); Non- Federal Airport Control Towers (NFCT); Systems Operations at the Air Traffic Control System Command Center (ATCSCC) and all field traffic management units; and when requested by the military ATC facilities. Bob Tarter Acting Vice-President ATO-Safety 8 06/24/08 7010.1T Appendix 1: Procedures for Evaluating AFSSs, FSSs and FCFSSs 1. DEFINITIONS. a. Action Plan. An action plan is documentation prepared by the Facility Manager that delineates corrective measures and an anticipated closure dates for items rated as “A.” This measure is required in addition to the three-step closure process. The plan may also be required for “P” and “E” (see definitions below) items that were not closed within the appropriate time frame. This item does not apply to FAA Contract FSS (FCFSS) facilities. b. Action Rating “A.” An action rating is assigned to any checklist or off-checklist item that is not accomplished in accordance with national, Service Area, or local requirements and the magnitude is such that it requires immediate attention (e.g., a safety issue). c. Appended Items. Appended items are new or elevated items identified in the course of conducting an evaluation at another facility or as a result of investigative findings obtained from an Operational Error/Deviation (OE/D), accident, incident, or other triggering event. d. Checklists. Checklists are used as minimum guidance in preparing for and conducting Full-Facility Evaluations (FFEs) and internal evaluations. e. Conformity Index (CI). Each on-site FFE conducted by Safety Evaluations must include a CI. The CI must essentially be the result of aggregating the weighted indices for each of the functional areas (System Safety, System Efficiency, and System Management) on the national checklist. System Safety is weighted more than the other functional areas. Instructions for calculating the CI are available from the National Statistics for Evaluations and Investigations (NSEAI) database. Safety Evaluations acknowledges that no two facilities are identical; therefore, CIs are not intended to compare facilities. The intent of the CI is to numerically depict a facility’s overall compliance with directives/regulations and to assist with identifying “at risk” facilities for non-compliance. f. Desk Audit. A desk audit is an off-site method of assessing checklist and offchecklist items. It is accomplished by the Safety Evaluations through discussion with facility personnel and/or review of requested tape recordings, data, and/or documentation. A desk audit is frequently used as a method of conducting Follow- Up Evaluations. 9 7010.1T 06/24/08 g. Follow Up Evaluations (FUE). An FUE is conducted on-site and/or through desk audit procedures to ensure that previously identified discrepancies were corrected. This evaluation must be conducted no earlier than 6 months after the date of the fullfacility evaluation. h. Full Facility Evaluation (FFE). An FFE is conducted on-site utilizing the appropriate national checklist to assess a facility’s overall performance. This evaluation is conducted every 2 years. i. Informational Rating “I.” An informational rating is assigned to any checklist or off-checklist item that may be of interest to readers. j. Internal Evaluation. This is defined as a facility's self-evaluation conducted by the Facility Manager's using the checklists and procedures outlined in this directive. k. Management Effectiveness Rating “E”. A management effectiveness rating indicates that management has not been effective in ensuring that discrepancies are resolved at the facility level. This item is intended to alert the Service Area Director or Contract Manager that assistance may be required. This rating will normally be assigned to a “P” item re-identified during subsequent evaluations. l. National Statistics for Evaluations and Investigations (NSEAI). NSEAI is a national database that contains evaluation checklists, evaluation reports, facility information, tracking control numbers, response data, and other statistical information for all flight service stations. m. Not Applicable Rating “N/A.” The N/A rating is assigned to any checklist item that does not apply to the facility being evaluated. n. Not Observed Rating “N/O.” The N/O rating is assigned to any checklist item that is applicable to the facility but is not observed during the course of the evaluation. o. Not Rated Rating “N/R.” The N/R rating is assigned to any checklist item that is applicable to the facility but for various reasons, such as time limitations, is not evaluated. p. Observed Event. An observed event identifies a situation witnessed by a member or members of the evaluation team that is determined to be operationally significant (e.g., a suspected OE/D). An observed event must be rated as “I” and described in the report. 10 06/24/08 7010.1T q. Off-Checklist. An off-checklist item is an assessed item that is not specifically identified on a national checklist. r. Problem Rating “P.” A problem rating is assigned to any checklist or off- checklist item that does not meet national, Service Area, or local requirements, and the item can be resolved at the facility level. s. Service Area Rating “R.” A Service Area rating is assigned to any checklist or offchecklist item that does not meet national or Service Area requirements and the item cannot be resolved at the facility. This rating applies only to the Alaska Service Area. t. Satisfactory Rating “S.” A satisfactory rating is assigned to a checklist item that is accomplished in compliance with national, Service Area, and local requirements. u. Special Evaluation. A special evaluation assesses specific areas, programs, offices, or organizations as directed by Safety Evaluations. v. Three-Step Closure Process. The three-step closure process is the required method by which items identified as “A,” “E,” and “P” must be resolved. This process does not apply to contract AFSS facilities. The required responses at 60 and 180 calendar days must describe the three steps as follows: (1) Corrective Action. The corrective action is the initial action or series of actions taken by the facility to correct the discrepancy. (2) Follow-up Action. The follow-up action is taken after an appropriate period of time to validate that the corrective action was successful. Documentation must include the date(s) that the follow-up action was accomplished and the results. (3) Management Control. The management control includes the action and/or program that will remain in place to ensure that the discrepancy does not recur. Additionally, the management control identifies the position(s) within the facility that have responsibility for the effectiveness of the management control and a schedule for periodic review. w. Closure Process for Contract Facilities. The contract office or the service provider will provide Safety Evaluations with a report of closed problems no later than 6 months after the full-facility report has been finalized. w. Washington Headquarters Rating “W.” A Washington headquarters rating may be assigned to any checklist or off-checklist item that does not meet national requirements and cannot be resolved at the Service Area or Contract Office level. A 11 7010.1T 06/24/08 “W” rating must not be assigned to any item without prior coordination with Safety Evaluations. 2. RESPONSIBILITIES. a. Safety Evaluations must: (1) Ensure that an annual evaluation program is developed and implemented. (2) Maintain a national database of evaluation information for analysis. (3) Provide a status report to the Executive Council each March and September for all open “R” and “W” ratings. (4) Review the evaluation process continuously to ensure its efficiency and effectiveness. (5) Review documentation on closed problem areas. b. Washington Headquarters Air Traffic Organization Program Directors/Managers must: (1) Ensure timely resolution for those items elevated to the Washington headquarters level. c. Alaska Service Area: (1) Ensure timely resolution for those items elevated to the Service Area. (2) Review responses from field facility managers addressing the actions taken to correct all “A,” “P,” and “E” items identified or appended during FFEs, FUEs, and special evaluations. The Service Area Director must prepare an endorsement indicating concurrence or non-concurrence with the manager’s actions for each item and determine whether Service Area assistance is required for “E” items. (3) When resources permit, provide personnel to participate in Safety Evaluations. d. FAA Facility Managers must: (1) Promptly initiate steps to correct “A” items when notified by the lead evaluator in accordance with paragraph 5a. (2) Prepare a response addressing measures taken to correct all “A,” “P,” and “E” items identified during or appended to FFEs, FUEs, and special evaluations. Responses must be prepared and submitted in accordance with paragraph 5. (3) When resources permit, provide personnel to participate in Safety Evaluations. 12 06/24/08 7010.1T (4) Furnish the lead evaluator, upon arrival at the facility, the total traffic count numbers for each calendar year since the previous FFE. (5) Be encouraged to conduct an internal evaluation every 2 years using the applicable FFE checklist. The goal of this activity is proactive Quality Assurance. 3. EVALUATION PROCESS. a. FFE. (1) Preparation and Notification. An FFE, utilizing the appropriate checklist(s) in the NSEAI database, will be conducted at facilities determined by Safety Evaluations. Safety Evaluations will notify the Facility Manager and Service Area Director or Contract Manager prior to conducting an FFE. (2) In briefing. An in briefing is conducted for the purpose of introductions and should include a short discussion of anticipated evaluation activities while onsite. (3) Conducting the Evaluation. Evaluators must conduct the FFE using all or some of the following methods: direct observation, position and/or tape/data monitoring, observation of training activities, review of administrative records, and interviews. To avoid unwarranted “N/O” ratings, evaluators will use every means available to verify items not readily observable. Interviews will normally be conducted with managers, supervisors, support specialists, union representatives, employee participation group representatives, and other facility personnel who volunteer. (The interview process will be limited to Federal Aviation Administration (FAA) Automated Flight Service Station (AFSS)/Flight Service (FSS) facilities.) Additionally, representatives from adjacent ATC facilities, FAA and non-FAA offices (customers, fixed-base operators, airport management personnel, etc.) may be interviewed. (4) Operational Error/Deviation (OE/D) Causal Factor. Safety Evaluations, in coordination with the Office of Aerospace Medicine’s Human Resources Research Division, AAM-500, analyzes completed FAA Form 7210-3, Final OE/D Reports, to compile statistics and determine trends regarding the causal factors for OE/Ds. Based on that analytical information and as a Quality Assurance initiative to further reduce the potential for OE/Ds system-wide, Safety Evaluations has identified certain checklist items that correlate to OE/D causal factors. When a problematic rating is assigned to any of these items, it must be concluded that the facility’s potential for experiencing an OE/D is increased. The evaluation report must be annotated to clearly state the correlation. (5) Daily Briefing. The lead evaluator will normally provide the Facility Manager or designee with a daily briefing on the progress of the evaluation. (6) Out briefing. The Facility Manager or designee must be briefed on the evaluators’/evaluation team’s findings at the conclusion of the evaluation. This may take place at the facility or via telephone conference. 13 7010.1T 06/24/08 (7) Re-identified Items. Items that are re-identified as “A,” “R,” or “W” during FFEs must retain that rating. A re-identified “P” item may be rated as “E.” If an “E” rating is used for a re-identified problem, a new tracking control number must be assigned an “E” rating. The appropriate closure process is required. b. FUEs. (1) Preparation and Notification. An FUE is conducted through an unannounced or minimum notification on-site evaluation, desk audit, or a combination of the two methods. An FUE will normally be conducted no earlier than 6 months after the date of the FFE out briefing or as determined by Safety Evaluations. Facility management may be requested to provide data for pre-evaluation review. The same process, as outlined in paragraphs 3a (2) through (7), must be used for on-site FUEs. (2) Reopened Items. When discrepancies are reopened during the FUE, the original tracking control number must be retained. The format in Figure 1 must be used to change the rating of a reopened item and to identify the evaluation process used to modify the rating. An item that is reopened as “A” during an FUE must retain that rating. Reopened “P” items may be assigned “E” ratings. If an item is rated as “E” during an FUE, the “E” rating will take the place of the “P” rating in the tracking control number (e.g., 02-T-XYZ- O1P would become 02-T-XYZ-01M). The appropriate closure process is required. (3) Open Items. Items previously rated as “A,” “P,” and “E” must be considered open if the appropriate closure process has not been identified and/or the discrepancy can be detected. Each open item must be addressed in the evaluation report with an explanation as to why the item was determined open. (4) New Items. New items identified during an FUE may be rated as “A,” “P,” “R,” “W,” or “I” as appropriate. The appropriate closure process is required. (5) Closed Items. Items must be considered closed when the discrepancy can no longer be detected and the appropriate closure process has taken place. c. Special Evaluations. (1) Preparation and Notification. Safety Evaluations must coordinate with the requesting office and notify the subject facility or organization through the appropriate manager. (2) Conducting the Evaluation. The in briefing, evaluation, and out briefing must be conducted at the direction of Safety Evaluations and the requesting office. (3) Tracking Control Number. The format in Figure 1 must be used for assigning tracking control numbers. 14 06/24/08 7010.1T d. Internal evaluations. Facility Managers are encouraged to conduct an internal evaluation every 2 years using the FFE checklist applicable to his or her type of facility. The goal of this activity is proactive Quality Assurance. e. Appended Items. Coordination with the Service Area Directors or Contract Managers and facility managers must be accomplished prior to appending a new item to the FFE report. A separate report identifying the appended item(s) must be forwarded to the Facility Manager. The appropriate closure process is required. The format in Figure 1 must be used for assigning tracking control numbers. Under the following circumstances, new items and previously identified “P” items elevated to an “E” may be appended to the most recent FFE report: (1) While monitoring interfacility operations during evaluations. For example, a problem may be identified at one facility while evaluating another. (2) As a result of investigative findings emanating from an OE/D, accident, or incident. (3) As a result of a facility submitting a third late OE/D report. 4. EVALUATION REPORTS. a. Report Completion. Results of all evaluations and audits must be documented to ensure that Washington headquarters, Service Area Directors, and Contract Managers remain fully informed regarding the effectiveness of the Air Traffic system. All final reports must be completed and distributed in a timely manner. To the extent possible, reports must be written in past tense. b. FFE Reports must: (1) Describe the results and findings of the evaluation or audit in a narrative format. (2) Assign tracking control numbers to all items identified as “A,” “P,” “E,” “R,” and “W” in the report in accordance with Figure 1, Tracking Control Number. 15 7010.1T 06/24/08 Figure 1: Tracking Control Number Tracking Control Number Example: 03-S-XYZ-01P-AE “03” refers to the year of the evaluation. “S” designates the type of facility: 1. S = AFSS/IAFSS/FSS/FCFSS “XYZ” is the facility identifier “01” is the tracking number, and “P” is the rating: 1. A = Action Rating 2. E = Management Effectiveness Rating 3. P = Problem Rating 4. R = Service Area Rating 5. W = Washington Headquarters Rating “AE” indicates problem identification other than during an FFE. The following must be used for appended items, evaluations conducted via special evaluations, and on-site follow-up evaluations: 1. AE = Appended Item 2. DA = Follow-up Evaluation conducted via desk audit 3. SP = Special Evaluation 4. U = Follow-up Evaluation (3) Be distributed as follows: (a) The original signed report must be sent to the Service Area Director or Contract Manager. (b) Copies of the report must be provided to Safety Evaluations and the Facility Manager. c. FUE Reports. (1) Item Classification. FUE reports must contain the status of all problematic items identified during the previous FFE and any appended items. Items must be categorized as reopened, open, new, or closed. Each item must contain a tracking control number and title followed by a description or explanation of findings. To the extent possible, reports must be written in past tense. (2) Reopened Items. Reopened items must be documented in accordance with paragraph 3b (2). (3) Open Items. Open items must retain their original tracking control numbers and be documented in accordance with paragraph 3b (3). (4) New Items. As necessary, use the format in Figure 1 for assigning tracking control numbers. Continue numbering new items sequentially from those reported in the facility’s most recent FFE. (5) Closed Items. Items closed via the appropriate closure process and those items closed during the FUE must be documented accordingly. (See paragraph 5, Responses.) (6) Report Distribution. The original report must be sent to the Service Area Director or Contract Manager. One copy of the report must be provided to each of the following: Safety Evaluations and the Facility Manager. d. Special Evaluation Reports. 16 06/24/08 7010.1T (1) Report Content. Safety Evaluations and the manager who requested the evaluation must determine the areas to be evaluated and the report format. If a special evaluation is conducted at a field facility, the findings must be documented in NSEAI. (2) Report Distribution. Safety Evaluations must make appropriate distribution. 5. RESPONSES. Responses to Safety Evaluations are required for all items rated as “A,” “E,” “R,” and “P.” This process does not apply to contract AFSS facilities. Responses must comply with the three-step closure process using the automated response process within NSEAI. If a facility is unable to utilize the automated response process, the Facility Manager must respond via memorandum using the format in paragraph 6. In addition, the following criteria apply: a. Action Plan. Proposed action plans for “A” items must be presented via telephone conference to a Safety Evaluations Manager and the Service Area Director, or designee within five calendar days after notification/identification of the “A” item. The purpose of the telephone conference is to obtain concurrence from the appropriate Safety Evaluations Manager and the Service Area Director. Action plans must delineate corrective measures and include an estimated date of resolution. A sample format is listed in paragraph 7. Following concurrence, the Facility Manager must ensure that the action plan is entered into NSEAI no later than 15 workdays after the out briefing. b. First Response. The first response must be received at Safety Evaluations no later than 60 calendar days after the date of the FFE or FUE out briefing. All “A” items must be closed by the date of the first response, and the action plan and three-step closure process utilized must be included in this response. Additionally, at a minimum, corrective action must have been initiated for all “E” and “P” items and documented in the first response. c. Second Response. The second response must be received at Safety Evaluations no later than 180 calendar days after the date of the FFE or FUE out briefing. All “E”, “R” and “P” items must be closed by the date of the second response, and the threestep closure process utilized must be documented in this response. d. Service Area/Washington Headquarters Response. Items rated “R” and “W” must be responded to at the discretion of Safety Evaluations. e. Special Evaluations Responses. Special evaluation responses must be at the discretion of Safety Evaluations. f. Alaska Service Area. Safety Evaluations will be responsible for conducting evaluations at AFSSs. The Alaska Service Area will retain the responsibility for 17 7010.1T 06/24/08 conducting evaluations at the FSSs and will use procedures identified in this Appendix. 6. EVALUATION RESPONSE FORMAT. Use the following format to respond to items specified in paragraph 5: Subject: INFORMATION: Full-Facility Evaluation, (City) ATCT (XYZ) From: Air Traffic Manager, (City) AFSS To: Manager, Safety Evaluations Date: Reply to Attn. of: The following steps have been taken for each action, management effectiveness, or problem rating identified during the most recent audit/evaluation by Safety Evaluations. a. (03-S-XYZ-01A) (Tracking control number and title of the action item as it appears in the evaluation report.) (1) Action Plan: (Briefly describe the plan that, in conjunction with the Service Area Director, was presented to and gained Safety Evaluations concurrence. Include the date of the telephone conference at which concurrence was attained.) (2) Corrective Action: (Describe completed corrective action(s). Include the completion dates.) (3) Follow-up Action: (Describe the follow-up action(s) accomplished to verify the success of the corrective action(s). Include the dates for completed follow-up actions and/or the planned completion date for any pending follow-up action.) (4) Management Control: (Describe the management control(s) implemented to preclude recurrence of the cited action item.) STATUS: We consider this item (open/closed). Note: The three-step process for closing an action item has to be completed within 60 days of the action item’s identification. b. (03-S-XYZ-02M) (Tracking control number and title of the management effectiveness item as it appears in the evaluation report.) (1) Corrective Action: (Describe completed corrective action(s). Include the completion dates.) 18 06/24/08 7010.1T (2) Follow-up Action: (Describe the follow-up action(s) accomplished to verify the success of the corrective action(s). Include the dates for completed follow-up actions and/or the planned completion date for any pending follow-up action.) (3) Management Control: (Describe the management control(s) implemented to preclude recurrence of the cited management effectiveness item.) STATUS: We consider this item (open/closed). c. (03-S-XYZ-03P) (Tracking control number and title of the problem as it appears in the evaluation report.) (1) Corrective Action: (Describe the corrective action(s) completed to date. Include the completion dates.) (2) Follow-up Action: (Describe the follow-up action(s) accomplished to verify the success of the corrective action(s). Include the dates for any completed follow-up actions and/or the planned completion date for any pending follow-up action.) (3) Management Control: (Describe the management control(s) implemented to preclude recurrence of the cited problem item.) STATUS: We consider this item (open/closed). (Air Traffic Manager's signature) 7. ACTION PLAN FORMAT. Use the following format to respond to action items as specified in paragraph 5a: Subject: INFORMATION: Action Plan(s), (City) AFSS (XYZ) From: Air Traffic Manager, (City) AFSS To: Manager, Safety Evaluations Date: Reply to Attn. of: In accordance with Order 7010.1, the following action plan(s) is (are) submitted for each action rating that was identified during the most recent evaluation by Safety Evaluations. Each action plan was coordinated with the Service Area Quality Assurance Manager. Safety Evaluations’ concurrence with the action plan(s) was gained during a telephone conference on (date). a. (03-S-XYZ-01A) (Tracking control number, title, and narrative for the action item exactly as cited in the evaluation report.) 19 7010.1T 06/24/08 Corrective Action: (Document corrective action(s) accomplished, the date of completion, any further planned corrective action(s), and the deadline(s) for completion. Remain aware that any action item must be closed, via the three-step closure process, within 60 days of identification.) b. (00-S-XYZ-02A) (Sequentially by tracking control number, continue the format described above for each action item.) (Air Traffic Manager’s signature) 20 06/24/08 7010.1T Appendix 2: Procedures for Auditing Non-Federal Contract Towers (NFCT) The review of Non-Federal Contract Tower (NFCT) operations by the FAA is no longer called an “evaluation,” but rather an “audit.” Safety Evaluations will conduct an audit of all NFCTs at least once every 3 years, beginning October 2006. These audits normally last approximately 8 hours or less and will cover 40-50 checklist items that deal with operational issues only. Safety Evaluations will not be reviewing the facility’s training, quality control, or administrative processes. The lead auditor will normally have someone assisting him/her and will provide the Facility Manager with an inbriefing and an exit briefing. During the exit briefing the lead will notify the facility manager of the status of all items relevant to the facility. These checklist items will be rated using the same criteria as FAA and FCT audits as follows: “M” Meets Requirements. This rating is assigned to a checklist item that is in compliance with national, Service area and local requirements. “N” Needs Improvement. This rating is assigned to a checklist item that complies with some national, Service Area, and local requirements, though several instances of variance from the standard are observed. This rating requires comment, a mitigation plan, and follow-up. “D” Does Not Meet Requirements. This rating is assigned to a checklist item that does not comply with national, Service Area and local requirements. This rating requires comment, a mitigation plan, and follow-up. “A” Requires Immediate Action. This rating is assigned to a checklist item with materiality to safety that requires immediate action. Immediate is defined as commencing on the same day that the item rating is assigned or received. This rating requires notification of the Service Area and Service Unit to obtain mitigation plan concurrence. “N/A” Not Applicable. This rating item is assigned to any checklist item that does not apply to the facility being evaluated. All items that do not meet requirements must be corrected in writing by the manager. Within 15 days of having the audit approved, managers are required to forward a mitigation plan for correcting each deficient item identified. These plans are to be sent to Safety Evaluations by the following means: 1. By e-mail, the plans may be sent to: (joe.auditor@faa.gov). NOTE: The Evaluations Auditor will provide the active FAA email address to the facility manager prior to audit completion. 2. By fax to 817-838-1930, Attention ATO-S Safety Assurance, or 3. By mail to: Department of Transportation 21 7010.1T 06/24/08 22 Federal Aviation Administration Southwest Region Headquarters Fort Worth Evaluation Group, AJS-32 Fort Worth, TX 76193-0032 Non-federal facilities cannot be granted access to the FAA Intranet; therefore, entries into the Facility Safety Assessment System (FSAS) database program will have to be accomplished by Safety Evaluations. NFCT facilities will be contacted upon receipt that your audit was approved. Upon that notification, the 15 day time limit to provide a written plan begins. After the mitigation plan is received, it will be entered into the FSAS database for review by the appropriate service area. Facilities will be notified when word that the plan was either approved or that the service area recommends additional mitigation. Facilities must also submit a status report every 30 days after the mitigation plan has been entered into FSAS. When the facility manager is satisfied that the item in question now “Meets Requirements”, notify Safety Evaluations in writing, using the same means outlined above, that the issue is considered closed and explain the steps taken. At that point, the rating will be updated in the FSAS database and the facility will be notified as such. Once the new rating is entered into FSAS, the item is considered closed and no further reporting is required.