|
|
This paper was originally presented at the 45th International Air Safety Seminar held at Long Beach, California in 1992. It was published in the Proceedings of the 45th International Air Safety Seminar (1992), Flight Safety Foundation, Arlington, Virginia, USA. A similar paper was published under the title "Data, Decisions, and Cockpit Technology" by the Society of Automotive Engineers (SAE) as SAE Technical Paper No. 922049. Additionally, it was included in the Report of the Flight Safety, Human Factors and Accident Prevention Regional Seminar and Workshop conducted in December 1993 at Rio de Janeiro, Brazil by the International Civil Aviation Organization (ICAO). It was also translated and published in Japan Air Lines' internal flight operations magazine.
During the past two decades, avionics advances have brought increasingly sophisticated technology into the cockpit. These advances were intended to provide benefits in both operating economics and safety. While many of the economic rewards have been realized, the safety advantages have yet to be proved conclusively. Some are concerned that today's automated systems may actually have negative effects on safety. The effectiveness of today's automation is largely a result of rapidly improving design of modern avionics systems. Where avionics design must make tradeoffs, or when designs prove to be less than optimum, flight crew training is called upon to take up the slack. Where both designer and trainer fail, safety is left to the crew and their airmanship skills. Researchers and industry experts have expressed the need for a high level philosophy to guide the development of effective cockpit automation. As these philosophies emerge, they agree on the need for the flight crew to be at the center of the operational loop. This paper presents an operational philosophy proposing the design of automated systems whose capabilities are organized into "levels-of-service" that may be "ordered" by the crew. Service-based training would encourage the crew to "purchase" required automation services using workload as a medium of exchange, with the goal of "earning a profit" from the transaction.
Avionics advances made over the last two decades have been promoted as beneficial based on two primary assumptions: that they provide economic benefits to the aircraft operator; and they improve flight safety.
Improved economics were to be derived from a combination of factors. Through automation of the crew's tasks, crew size could be reduced, cutting personnel costs. Additionally, the new generation of solid state avionics and display systems would deliver a dollar saving combination of improved reliability and reduced maintenance. Finally, automated monitoring and recording of systems parameters would provide trend monitoring capabilities, saving money in the maintenance of high value systems such as engines and APUs. Today's advanced avionics have delivered on most, if not all, of these economic promises, but their safety effectiveness has been a subject of dispute.
Proponents of automation in pursuit of improved safety have pointed to several avenues by which this would occur. First, automation would perform the routine, lower level work of aircraft operation, freeing the crew to perform higher level tasks such as monitoring flight progress, dealing with problems, and making decisions. Such systems would allow crew members to be systems managers, rather than systems operators. Also, automated access to all relevant flight and systems data would improve crew decision making by making sure that they had the best basis for those decisions.
After more than a decade of experience with these advanced systems, operators and researchers are finding that the promise of improved flight safety is still largely unfulfilled. Some observers contend that increased automation may actually be creating new hazards. The automation issues which impact safety include flight crew workload, avionics failure modes, degradation of basic piloting skills, and incompatibilities of cockpit systems with the Air Traffic Control system.
Aviation researchers have discovered disturbing trends among the crews of the new generation of automated aircraft. Among those are aberrations in the ways in which flight crew workload is affected. Remembering that automation was supposed to reduce workload to free the crew to perform higher level tasks, they are finding that most workload reductions are occurring when work levels were already low, such as during cruise. As workload is decreased, there seems to be an insidious trend toward increased complacency, lack of vigilance and even boredom among the crews of highly automated aircraft. In historically high workload situations, such as departure and arrival, automated systems can actually increase crew activity, detracting from critical vigilance for outside traffic and awareness of position, terrain, and the general ATC situation.
As digital avionics were first being introduced, one of the advantages claimed for them was their easily recognized failure modes. Unlike electro-mechanical instruments, it was supposedly unlikely to have a difficult to detect, un-annunciated failure. Digital logic simply would not permit it. In practice, errors in digital systems can be surprisingly subtle. As Dr. Earl L. Wiener (1988) describes using an alarm clock analogy, we have traded the potential for an undetected 5 or 10 minute error in setting the analog alarm clock for a very precise 12 hour error (pm vs. am) in setting today's digital alarms.
Recognition and recovery from automation failures can prove to be very difficult and involve very high workloads. The recent series of very slow, almost imperceptible, roll malfunctions experienced on Boeing 747s demonstrates that recognition and recovery can be slow and late. Another example involves a particular version of software on a leading transport that required five separate conditions to exist before a specific problem would manifest itself. Additionally, research has shown that many crews are reluctant to override an automatic system, even though there are obvious discrepancies in data being presented to the crew.
There is growing concern that automated systems can fail in ways that are both unanticipated and untrained. This difficulty in detecting system errors requires the crew to cross-check primary flight and navigation displays to ensure proper performance of the automated systems. This type of monitoring has the potential for giving new meaning to the concept of "Raw Data". Crews are required to mentally "fly" the aircraft using raw data inputs as a monitoring technique for automatic systems.
There is growing, but still unsubstantiated, concern regarding degradation of pilot skills and proficiency through the use of extensive automation. Flight crew concern may be an effective counter to the development of problems in this area. Recent research (Wiener, 1992) shows that while a majority of pilots are concerned about skill deterioration when flying automated aircraft, only a minority believe their skills have been affected. Carefully designed standard operating procedures can play an effective role in maintaining proficiency in routine operations.
While the cockpit is receiving a high level of attention from the makers of advanced avionics, the Air Traffic Control system has not escaped attention. Again, the rational is reduced workload, and improved safety, but the effects of ATC automation will ripple into the cockpit just as much as the reverse has already begun.
Among the systems ripe for conflict is the TCAS system. How does a flight crew respond when faced with a conflict between an ATC clearance and what his on-board TCAS commands? And worse, what other conflicts may be created in the effort to fly safely.
It is obvious that the designers of aviation automation systems bear tremendous responsibility for the effectiveness and safety of the automated cockpit. But aviation trainers also play a role, as Dr. Stanley N. Roscoe (1980) asserted, saying that training must "complete the job left undone by the engineer." Even when the trainer has completed his job, there is a third line of defense for ensuring safety of flight, the crew's native experience and airmanship. When all levels fail; design, training, and airmanship; accident risk increases (Figure 1).
The horrors of inadequate integrated design may be seen in exaggerated form in the front seat of some police patrol cars. There you will find a bewildering array of radio, computer, data link, signaling, video recording, radar, weapons, and protective equipment; none of it intended to work as an integrated system, and little of it intended to be operated by a driver traveling at high speed in a dynamic, congested environment.
Indeed, many police departments have resorted to specialized training on the use of the simple hand-held microphone while in a high speed chase. A peculiar hand-sliding steering technique is taught to avoid wrapping the cord around the steering column while maneuvering rapidly. Training and awkward techniques have been used to compensate for an inefficient and possibly dangerous design.
The first line of defense against many of the automation problems discussed above lies with the system designer. There is an important distinction between the successful design of a component such as a navigation receiver and the successful design of an integrated cockpit system that combines many of the diverse functions of aircraft system operation. Today, transport systems engineers are taking the automation issues head on, providing design improvements such as glareshield mounted controls and indicators to reduce heads down time while reprogramming flight management systems. Additional research and simulator testing is also in progress to address other automation issues.
During the cockpit design process, training effectiveness could also be considered as a potential design driver. Design decisions between two seemingly equivalent options could be based on selecting the design option requiring the least training, or the one requiring the lowest level training device. Current airframe design practice does not typically involve the trainer until after the design is frozen. At this stage, the designer fills the role of subject matter expert in the training development process. Training requirements are not typically fed back into the design process.
Cockpit Resource Management (CRM) is a powerful and widely accepted training tool available to a broad spectrum of the aviation community. Cockpit automation systems are treated in most CRM programs as resources that are available to flight crews in the effective management of their flight operations. Unfortunately, most training programs do not include in-depth training on automated avionics systems at the level required for crews to effectively manage these systems, even with the best CRM techniques.
Current avionics training is primarily procedural in nature, with little or no systems instruction. As a result, despite the promise that advanced avionics systems would allow crews to become managers instead of operators, most training courses are designed to accomplish just the opposite. Courses typically do not provide insight into the philosophy (if any) which drove the design, and seldom addresses the software aspects of the design. Often, training developers adopt the attitude that since the crew can not affect these matters, they have no need to know about them. The training development process must be enhanced to capture the knowledge that would permit the crew to gain the most benefit from automated systems. Without a formalized process, this is often accomplished in an unstructured way using undocumented techniques.
Current training programs also suffer from a sort of "Loss of Corporate Memory" telling why systems are designed in a particular way. When course materials are originally designed and developed, the original cadre of instructors gains significant insight through their direct interactions with the designers. This insight is frequently passed on informally during classroom discussions even though it is not included in the course materials. Soon, however, instructor turnover following courseware development causes this insight to be lost or distorted for succeeding crews. Just as the trainer can contribute to the design process, the designer should be an integral part of the training development team, contributing to the crew's understanding of the automated systems.
There are currently pioneering efforts underway to address training issues relating to cockpit automation. Delta Airlines has developed an Introduction to Aviation Automation (IA2) course that all pilots transitioning to glass cockpit aircraft are required to take. Lasting one half day, the class teaches Delta's automation philosophy and includes incident and accident discussions related to automation.
But what happens when both design and training leave the crew with gaps in their ability to deal with the in-flight environment? In the closing session of the NASA/MAC Cockpit Resource Management Workshop held at San Francisco in 1986, a senior British airline captain observed that the then-new CRM training concepts that were being discussed were really nothing more than airmanship. Using their airmanship skills, the crew provides resilience to ensure safety in an environment of change, uncertainty and error. Indeed, Dr. Charles E. Billings (1991) defines airmanship as "the ability to act wisely in the conduct of flight operations under difficult conditions." It logically follows that we rely on the flight crew's airmanship to overcome systems deficiencies and operational situations that have eluded both the design and training processes. Ultimately, the crew is responsible for the safety of each flight.
Several researchers and industry experts have expressed the requirement for an overall design philosophy to serve as the bedrock upon which effective cockpit automation systems are to be built. Wiener (1988) explored elements of automation philosophy. The Air Transport Association of America (1989) stated that "A fundamental concern is the lack of a scientifically based philosophy of automation which describes the circumstances under which tasks are appropriately allocated to a machine or a pilot." Billings (1991) presented the most comprehensive definition to date of the concepts and guidelines supporting what NASA refers to as human-centered aircraft automation (Figure 2). Dr. John K. Lauber, a Member of the National Transportation Safety Board, states "What is missing are principles, rules and guidelines defining the relationship between that technology and the humans who must operate it." (Phillips, E.H. 1992).
Standard Operating Procedures (SOPs) have been an effective tool used by aircraft operators for many years to present the operator's philosophy of safe and efficient flight operations. Scientifically based automation philosophies are useful in the formulation of operational guidance and training for flight crews. Operationally oriented automation philosophies can assist flight crews in wisely and effectively applying automated systems to the safe conduct of each flight.
Integrated design could incorporate a selection of information and automation "services" that are selected by the flight crew as the situation dictates. Crews would select "levels-of-service" based on workload and the situation. Services can be selected or changed to respond to ATC or other situational requirements. This allocation of resources is essentially a management decision and is dictated by dynamic operational conditions. These decisions should be controlled by the flight crew in real time. "Hard wired" solutions within an automated system may not be able to meet the demands of the highly volatile flight environment (Figure 3).
This service-based orientation of the use of automation would keep the flight crew at the center of the operational loop by allowing them to select from a spectrum of services that are provided by the aircraft, crew, and outside agencies. Requests for service would always originate with the crew. As they perform tasks, services would be "purchased" from the aircraft systems, enabling them to manage the accomplishment each task. The medium of exchange for these transactions would usually be workload. The crew would "invest" a certain amount of workload in system setup and operation. If properly selected, they would show a "profit" in the form of reduced overall workload, access to required information, or improved operational efficiency. This selection of levels-of-service is different from the selection of modes. Mode selection is accomplished at a given level-of-service, whereas a change in level-of-service may completely alter the modes available for use (Figure 4).
Under this concept, manual operation by the crew is the most basic level-of-service. As such, the crew must be prepared to provide services at any time. This means they must not only be prepared psychologically, but also from the standpoint of proficiency.
Avionics system design would benefit from implementation of the service-based concept in a number of ways. It would provide a structure for designs that would be inherently human-centered because they are configured from the start to provide assistance to the crew on-command, rather than acting to replace crew functions. Part of the design process would be definition of the levels-of-service which individual and integrated systems provide to the crew. This approach would provide a context for the resolution of issues relating to when automation serves the crew and when the crew serves the automation. An additional level-of-service would be a natural extension of this philosophy, that of individual preference of input and output. While some pilots might be most comfortable with displays consisting of graphic and pictorial data, others may find alphanumeric data most understandable. Similarly, keyboard input may be most efficient for some individuals while others would prefer menu-based or other systems.
A service-based training approach would provide a sophisticated and positive extension to the "Turn-It-Off" training approaches currently being discussed. Rather than training a crew to simply disconnect a system, they would learn to "purchase" alternative services that are more appropriate to the existing situation, either "cutting their losses" or earning a greater workload "profit".
The service-based approach to automation would require a thorough understanding of automation system design, function, and failure modes, as well as operating procedures. This means teaching the structure of both hardware and software, giving the crew the background needed to support decisions relating to effective level-of-service selection and utilization. Training would also include coverage of avionics updates and software revisions in the recurrent training environment.
To be effective, training must provide a common vocabulary for crews to use when discussing the selection, use and monitoring of automated systems. Using this vocabulary and the underlying concepts of automation usage, services-based training would provide a formal structure for development of training objectives related to automated systems. This in turn would enhance the ability of scenario designers to include automation related objectives in Line Oriented Flight Training and other lesson structures. It would also provide a context for instructors and check airmen when making decisions on the availability or denial of automated systems during training and checking.
Design changes are largely impractical for aircraft that are already in operation or are far advanced in the design cycle. This prohibits design changes to incorporate service-based features. It is still possible to implement a service-based approach to crew training and flight operations, however. Through analysis of the systems on these aircraft, the levels-of-service available for the automated systems would be identified and incorporated into training courses and operations philosophies. This would allow operators of diverse fleets to maintain a common approach to flight deck automation across all equipment, regardless of vintage.
In the end, the crew provides the final defense against accidents through their human ability to adapt, learn, and overcome adversity. This ability to use the human element to overcome change, uncertainty, and error is called airmanship. As we continue to design better aircraft and more effective training for their crews, we must always allow for this most vital of all contributors to flight safety, but we should never knowingly rely on it.
Many people have assisted me in formulating the ideas and gathering the data necessary for the development of this concept. I would particularly like to express my appreciation to Dr. Edwin Hutchins of the University of California, San Diego for sharing his views on flight deck automation, allowing me to sharpen the focus on several key elements of this approach. Others whose contributions are greatly appreciated include Dr. Earl L. Wiener of the University of Miami, Dr. Charles E. Billings at NASA Ames, Dr. H. Clayton Foushee at the Federal Aviation Administration, and Mr. Walter S. Coleman, formerly with the Air Transport Association of America and now with the Regional Airline Association.
Air Transport Association of America. (1989). National plan to enhance aviation safety through human factors improvements. Washington, D.C.: Author
Billings, C.E. (1991). Human-centered aircraft automation: A concept and guidelines (NASA Technical Memorandum No. 103885). Moffett Field, CA: NASA-Ames Research Center.
Cooper, G.E., White, M.D., & Lauber, J.K. (Eds.). (1979). Resource management on the flight deck (NASA Conference Publication No. 2120; NTIS No. N80-22083). Moffett Field, CA: NASA-Ames Research Center.
Diehl, A.E. (1989). Human performance aspects of aircraft accidents. In R.S. Jensen (Ed.), Aviation psychology. Brookfield, VT: Gower Technical Publishers.
Krey, N.C. (1988). Using Aircrew Coordination Training techniques to enhance warfighting capabilities. In Proceedings of the 10th Interservice/Industy Training Systems Conference. (pp. 115-120). Washington, D.C.: National Security Industrial Association.
Krey, N.C., & Rodgers, D.W. CRM for Part 91 and 135 operations. In Orlady, H.W., & Foushee, H.C. (Eds.). (1987). Proceedings of the NASA/MAC workshop on cockpit resource management (NASA Conference Publication No. 2455, pp. 158-169). Moffett Field, CA: NASA-Ames Research Center.
Krey, N.C. (1992) Data, decisions, and cockpit automation. SAE Technical Paper Series 922049. Warrendale, PA: Society of Automotive Engineers.
Lauber, J.K., & Foushee, H.C. (1981). Guidelines for line-oriented flight training. Vol. I and II. (NASA Conference Publication No. 2184). Moffett Field, CA: NASA-Ames Research Center.
Norman, S.D. & Orlady, H.W. (Eds.). (1988). Flight deck automation: promises and realities (NASA Conference Publication No. 10036). Moffett Field, CA: NASA-Ames Research Center.
Phillips, E.H. (1992). Pilots, human factors specialists urge better man-machine cockpit interface. Aviation Week & Space Technology, March 23, 1992. (pp. 67-68).
Roscoe, S.N. (1980). Aviation psychology. Ames, IA: Iowa State University Press.
Ruffell Smith, H.P. (1979). A simulator study of the interactions of pilot workload with errors, vigilance, and decisions. (NASA Technical Memorandum No. 78482). Moffett Field, CA: NASA-Ames Research Center.
Sarter, N.B., & Woods, D. D. (1991). Situation awareness: a critical but ill-defined phenomenon. The International Journal of Aviation Psychology, 1, (pp. 45-57).
Wiener, E.L. (1988). Cockpit Automation. In E.L. Wiener & D.C. Nagel (Eds.), Human factors in aviation. (pp. 433-461). San Diego, CA: Academic Press.
Wiener, E.L., Chidester, T.R., Kanki, B.G., Palmer, E.A., Curry, R.E., & Gregorich, S.E. (1991). The impact of cockpit automation on crew coordination and communication: Vol. I. (NASA Contractor Report No. 177587). Moffett Field, CA: NASA-Ames Research Center.