In 1983, the cognitive psychologist Lisanne Bainbridge published a foundational paper titled Ironies of Automation. Its main conclusion captured a contradiction inherent in the design of technical systems: by automating the tasks considered routine or predictable, the human operator is pushed out of the continuous process and is only required to step in during anomalous, unforeseen or critical situations. In practice, operators are asked to resolve the most technically complex scenario precisely when the system has deprived them of the daily practice that keeps their expertise alive.
For four decades, this phenomenon has been analyzed and regulated with precision in critical sectors such as commercial aviation, maritime navigation and anesthesiology. However, the current integration of probabilistic models and generative systems into analytical and organizational management tasks seems to be reproducing exactly the same paradox, ignoring the lessons consolidated by the human factors literature.
The mechanics of cognitive deconditioning (deskilling)
The sustained delegation of processes to automated problem-solving tools introduces two well-documented effects: automation bias and automation complacency. Operators tend to overestimate the accuracy of the system because of its apparent formal coherence, progressively reducing their critical attention and their verification effort. As source cross-checking, the structuring of interim reports and basic analytical calculation are transferred to the tool, a gradual loss of skill takes place (skill decay, or deskilling). Solving a complex problem requires constant familiarity with its simplest components. If teams do not regularly practice formulating hypotheses on their own, their ability to identify subtle biases or glaring errors in the software’s output declines significantly.
The need for a deliberate degraded mode
In aviation safety protocols, flight simulators are not used solely to monitor stable automated trajectories, but to train pilots to disconnect the system and manually recover control under extreme conditions. Operational expertise (manual proficiency) is not preserved through inertia; it requires structured practice of the manual procedure and of operating in degraded mode.
In the governance of most organizations, however, operational resilience is measured almost exclusively in terms of technological recovery time: how many minutes it takes for a service to be restored after an outage. The resilience of human teams is rarely audited or trained: how much real capacity staff retain to sustain operations and make judgment calls if the tool is unavailable or is no longer reliable.
Preserving autonomous skill is not a stance against technological innovation, but the only objective condition for oversight of the system to be effective rather than merely declarative. Practicing the ability to operate without the tool is an essential design requirement for any institution that intends to retain sovereignty over its own processes.
