Automation bias

Automation bias is a tendency for humans to favor suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct. This bias is a significant issue in industries and professions where the use of automated systems is common, such as in aviation and medicine.

Overview

Automation bias occurs when a person favors the advice or result given by an automated system, even when other sources or their own judgment might suggest a different outcome. This can lead to errors if the automated system is incorrect. The bias can be particularly problematic in high-stakes environments like aviation and medicine, where errors can have serious consequences.

Causes

The causes of automation bias are not fully understood, but several theories have been proposed. One theory suggests that people trust automated systems because they believe they are more reliable and less prone to error than humans. Another theory proposes that people may be more likely to trust automated systems because they are easier to use and require less mental effort.

Effects

The effects of automation bias can be serious, particularly in fields like aviation and medicine. In aviation, for example, pilots may rely too heavily on automated systems and fail to take corrective action when the system makes a mistake. In medicine, doctors may rely too heavily on automated diagnostic tools and fail to consider other possible diagnoses.

Mitigation

Several strategies have been proposed to mitigate the effects of automation bias. These include training people to be aware of the bias and to question the results of automated systems, designing systems to be more transparent about their decision-making processes, and implementing checks and balances to ensure that decisions are not made solely on the basis of automated advice.

See also

• Cognitive bias
• Decision-making
• Human factors and ergonomics