Marie-Pascale Allard & Jade Brassard
Sauguenay, Quebec
INTRODUCTION
Thousands of planes take off every day and their crew are never immune to mistakes. Imagine any flight with any conditions, where both pilots make x number of errors. Can it be said that other pilots, under the same conditions, can be better than their predecessors if they have taken the trouble to analyze their mistakes? We believe they would make just as many mistakes, but these would potentially be different. To verify this hypothesis, an experiment was carried out with reference to the LOSA report and with the help of piloting teachers.
Threat and error management
First, a threat is what is external to the individual, while a mistake is what is done by them. In the past, when a pilot made a mistake, his superiors would focus directly on the mistake. The pilot was no longer to reproduce the latter or would be subjected to reprimands. Today, with the establishment of a Threat and Error Management (TEM) procedure in the 1990s, aviation professionals are focused on identifying the threats that were present and that led to the error (Fédération française aéronautique, n.d. ; Arickx, 2016).
Context
The training in threat and error management is based on the data provided by the audit (document presenting the observations and recommendations of experts in the field (Linternaute, n.d.)) of line service security (LOSA). The LOSA program, issued by the International Civil Aviation Organization (ICAO) (Organisation de l’aviation civile internationale, n.d.), is made up of experts who, after observing more than 4,500 flights, issued a report (Organisation de l’aviation civile internationale, 2002). This included the threats that compromised safety, the way they were handled, the errors they led to, the way those were processed and the behaviors generally linked to accidents (Organisation de l’aviation civile internationale, 2002).
In terms of threats, the observers of the LOSA program have noted that the most recurrent ones are those caused by the weather, such as strong winds or thunderstorms, and by air traffic control, for example heavy traffic on the ground. Otherwise, they could also be threats caused by the aircraft, such as a breakdown in flight, or by human factors causing more or less stress, depending on the context, on the pilots. Of all threats present, 90% are well handled by the crew (Merritt & Klinect, 2006).
Regarding errors, 80% of flights contain at least one, the average being 3 per flight. An error can be, for example, a bad call, a role reversal or a wrong maneuver. 25% of errors are handled incorrectly. This percentage can be divided into two parts: 6% lead to another error and 19% directly lead to an undesirable aircraft situation (Merritt & Klinect, 2006).
With regards to undesirable situations, one third of flights present such a situation. An undesirable situation is a deviation in the position or speed of the aircraft (Marquis, 2014), that is, the aircraft is no longer in the state it was in the beginning. The most common are incorrectly configured systems (Merritt & Klinect, 2006). In an extreme case, the undesirable situation could be a plane crash.
METHODS
In this experiment, we will observe the apprentice pilots fly in a simulator and note any mistakes they make. This flight can be divided into seven major parts: engine start, takeoff, initial climb, level flight, hold pattern, instrument approach and landing. Take-off will be at Montreal airport (CYUL), the holding pattern above the Saint-Jean (YJN) VHF omnidirectional radio beacon (VOR) and landing at Saint-Jean airport (CYJN). Throughout the flight, subjects will encounter several threats. They can be classified into four categories: weather, communications, stress and aircraft threats (Traversy, G., Personal communication, October 8, 2019).
Presented threats
Students will fly at night and will be subject to weather conditions with crosswinds, low ceiling, low temperature (-22 ° C) as well as turbulence when cruising.
Regarding air traffic control (ATC) (Organisation de l’aviation civile internationale, n.d.), communications will be in English, which will bring unfamiliar terminologies. In addition, since subjects are used to flying around an airport where traffic is low and not high density like Montreal, they will have to use certain terms that they have rarely used before.
The largest obstacles for apprentice pilots during their flight will certainly be found in the category of human factors. Indeed, the stress level will be particularly high due to the fatigue at the end of the day, the presence of observers, the time constraint, and the unfamiliar environment in which they will find themselves.
On the aircraft side, some navigation systems are partially dysfunctional.
Hypothesis
In order to prove that as long as there are threats there will be errors, we will observe the first two simulator sessions without intervening. Any errors made will be noted. Before the last two simulators, we will communicate in detail all the notes taken during the previous sessions to the pilots of these flights. The tested assumption is that although they will not make the same mistakes, the pilots in Simulator Sessions 3 and 4 will make as many mistakes as those in Sessions 1 and 2.
Experiment
The experiment took place on November 12, 13, 19 and 20, 2019 in the MECHTRONIX ASCENT flight training devices of the Centre québécois de formation aéronautique (CQFA). The sessions were conducted under the supervision of Mr. Gilles Traversy (Traversy, G., Personal communication, October 8, 2019) and Mr. Daniel Bordeleau (Bordeleau, D., Personal communication, November 12, 2019), piloting teachers at the CQFA.
RESULTS
Following the sessions, we noted all the errors made and classified them according to a three-tier system. Yellow corresponds to a gesture which does not involve danger but which does not comply with the standards. The orange represents an action that could have had a larger impact, but did not. As for red, it is associated with errors that have led to an undesirable situation. The classification was carried out with the help of Ms. Karen Lindsay (Lindsay, K., Personal communication, October 8, 2019), also a flight instructor at the CQFA, and Mr. Gilles Traversy (Traversy, G., Personal communication, October 8, 2019) (see Annex 1).
During the first two flights, it is observed that it was during the initial climb and the instrument approach that the errors classified in red were made. On the climb, both crews displayed their course bar incorrectly and did not meet altitude standards. During the approach, errors regarding the altitude standards were still made. In addition, the commanding officer and the first officer did not display the same altimeter settings and it was near this point that the transfer of leadership took place. As for the errors labeled yellow and orange, they are mainly related to role reversals, primarily during engine start-up, and poorly completed checklists.
For the two successive flights, there were far fewer errors in terms of cockpit management and checklists. In fact, during the engine start-up phase and the take-off phase, the errors are fewer and of lesser importance. However, as soon as the aircraft leveled off, the errors were very common and many were classified as red. In addition, the faults of the course bar, altimeter setting, and transfer of leadership were repeated by the last two crews.
DISCUSSION
We explain the considerable reduction in errors in the execution of procedures, errors in yellow, by the fact that the last four pilots had received the detailed notes taken from the previous flights. Indeed, when the candidates for the last two flights arrived, they appeared to be much more confident than their predecessors. Thus, it was possible to observe flight beginnings much closer to those performed by professionals. Even so, this is the only improvement we have seen between the first two and the last two. Indeed, in the middle of the session, their stress level outweighed their confidence. As a result, in total, they made as many mistakes as the first two crews. Since the course bar and altimeter setting error have been made four times, it is likely a lack of training. As for the leadership transfer, it was certainly the threat of stress that caused it. Indeed, the stress was much higher on the commander, which caused the first officer to take up more and more space.
During the experiment, an important source of error interfered with the results. It turned out that one of the candidates for the fourth flight had not reached a level of learning equivalent to that of the others. This explains why from the beginning of the approach the errors multiplied, and it is also what would have led to the confusion of the first officer about the position of the aircraft.
In conclusion, we confirm our hypothesis because the last four pilots made as many mistakes as the first four. On the other hand, we notice an imbalance in that they made fewer errors at the beginning but more at the end of the flight. Experimentation showed us that, in order to reduce the number of unwanted situations, managing threats would have been much more effective than managing errors, as recommended by the TEM model. This model, which is used in major airlines, is presented to students as part of the Crew Flight course. After having had a brief overview of the teaching methods used with students, we noticed that this philosophy of dealing with threat instead of error is not applied by teachers in an educational setting. Yet, for a student to improve, would it not be much more effective to tackle their outside environment rather than the very mistake they made? And maybe this way of looking at this problem, which has come a long way in the medical field, should now be incorporated into the whole field of education.
ANNEX I
YELLOW: non-compliant gesture, but without danger
ORANGE: action that could have had repercussions, but did not
RED: action that led to an undesirable situation
ANNEX II
YELLOW: non-compliant gesture, but without danger
ORANGE: action that could have had repercussions, but did not
RED: action that led to an undesirable situation
REFERENCES
Arickx, D. (2016, June 6). Traitement des erreurs et des menaces. Airfrance. https://www.acat-toulouse.org/uploads/media_items/threat-and-error-management-tem.original.pdf
Bordeleau, D., Personal communication, November 12, 2019.
Fédération française aéronautique. (n.d.). Gestion des risques. Clé Fi. http://dev.clefi.fr/ressource/TEM.pdf
Lindsay, K., Personal communication, October 8, 2019.
Linternaute. (n.d.). Audit. In Linternaute.fr. https://www.linternaute.fr/dictionnaire/fr/definition/audit/
Marquis, P. (Ed.). (2014). Sécurité aérienne – Nouvelles. Transports Canada. http://publications.gc.ca/collections/collection_2014/tc/T12-9-2014-2-fra.pdf
Merritt, A., & Klinect, J. (2006, December 12). Defensive flying for pilots: An introduction to threat and error management. Flight Safety Foundation. https://flightsafety.org/files/tem_dspt_12-6-06.pdf
Organisation de l’aviation civile internationale. (n.d.). Au sujet de l’OACI. https://www.icao.int/about-icao/Pages/FR/default_FR.aspx
Organisation de l’aviation civile internationale. (2002). Audit de sécurité en service de ligne (LOSA). Autorité de l’aviation civile. https://aacrdc.org/bibliothequeAAC/bibliotheques/OPS/9803%20LOSA%201e%20EDITION%20%20FR.pdf
Traversy, G., Personal communication, October 8, 2019.
ABOUT THE AUTHORS
Marie-Pascale Allard
Marie-Pascale is a student from Saguenay, Quebec. She graduated from high school in 2020. She was a student at Odyssée Lafontaine / Dominique-Racine in the Excellence program. This program focused on science allowed her to realize the Expo-sciences project during these five years. The success of these numerous projects allowed her to win several awards, notably the Consortium de recherche en exploration minérale (Mineral exploration research consortium) and the Manufacturier innovant du CRIQ (CRIQ's innovative manufacturing) award. During her last project, she and her teammate conducted an experiment to learn more about threats and errors in aviation.
Jade Brassard
Jade is a student from Saguenay, Quebec. She completed her high school education at École de l'Odyssée Lafontaine / Dominique-Racine, where she was enrolled in the Excellence program. This program, which focuses on science, supervises its students in the realization of a science fair project. Each year, Jade has carried out a science fair project, some of which have won her awards, notably the Consortium de recherche en exploration minérale (Mineral exploration research consortium) and the Manufacturier innovant du CRIQ (CRIQ Innovative Manufacturing) awards. In her final year, she and her teammate Marie-Pascale Allard conducted an experiment on threat and error management in aviation. Unfortunately, this research was never presented due to the COVID-19 pandemic.