---

---

Introduction

Human-in-the-loop (HITL) processes are widely promoted as a way to combine algorithmic efficiency with human judgment. In principle, human oversight is meant to catch errors, apply contextual reasoning, and compensate for known limitations of automated systems. In practice, however, evidence suggests that human involvement may not reliably deliver these benefits and can introduce new, systematic failure modes.

A recent paper by Beck et al., Bias in the Loop: How Humans Evaluate AI-Generated Suggestions, provides a clear illustration of this problem. The study shows that human oversight does not automatically mitigate AI errors. Instead, cognitive biases shape how people interpret and respond to AI-generated suggestions, often leading to over-reliance, reduced scrutiny, and the acceptance of incorrect outputs. Rather than acting as a corrective layer, human judgment can become another source of error in the system.

These biases are not new. What has changed is the range of contexts in which they now operate. AI has moved human–machine decision-making from specialised, high-stakes domains such as aviation and medicine into everyday tools, productivity software, and routine decision-support workflows. In these environments, human-reviewed outputs may feed back into the training of future systems, creating feedback loops that reinforce and amplify initial mistakes.

Understanding how people evaluate AI outputs and how cognitive biases shape that evaluation is therefore central to designing HITL workflows that actually achieve their intended goals of accuracy, efficiency, and safety. Beck et al.’s findings sit within a broader body of research examining the cognitive shortcuts people take when interacting with automated systems. The section that follows synthesises (some of) the more widely discussed biases in the literature and examines their implications for human–AI collaboration.

Three mechanisms are particularly well supported in the literature: automation bias, algorithmic loafing, and automation-induced complacency. Although described separately, they share a common dynamic which is that the presence of automation reduces cognitive engagement at precisely the moments when careful judgment is required.

Automation bias

Automation bias is one of the most extensively documented failure modes in human–AI interaction. It refers to the tendency to over-trust automated systems, leading people to defer to machine-generated suggestions even when those suggestions are incomplete or incorrect. This bias manifests in two systematic error patterns.

The first is errors of omission, where operators fail to notice problems because the automated system does not flag them. In aviation, such omissions have contributed to incidents in which flight crews overlooked critical changes in altitude or system state simply because no alert was issued. Controlled experiments reproduce the same effect: when monitoring aids intentionally withhold prompts, operators frequently miss significant changes, in some cases more than half the time. The absence of a signal is implicitly treated as confirmation that everything is functioning as expected.

The second pattern is errors of commission, where users actively follow incorrect automated advice despite contradictory evidence. In simulated flight tasks, participants have been shown to follow faulty automated recommendations even when other indicators clearly conflict with them. Some studies report that a majority of users commit multiple commission errors within a single session. These findings demonstrate that automation can override not only vigilance but also explicit knowledge and perceptual cues.

Together, omission and commission errors demonstrate that a system’s outputs can be weighted more heavily than other sources of information, including the user’s own judgment.

Algorithmic loafing

Where automation bias describes what errors people make, algorithmic loafing helps explain why cognitive engagement declines. Analogous to social loafing in group work, algorithmic loafing refers to the tendency for individuals to expend less mental effort when an AI system is involved in a task and treated as a competent teammate. Rather than critically evaluating AI outputs, users shift into a mode of passive acceptance.

Experimental work by Inuwa-Dutse et al. demonstrates how algorithmic loafing emerges in practice. In their studies, participants were exposed to sequences of correct AI decisions, establishing the system as a highly competent collaborator. This perceived competence led participants to engage less deeply with subsequent recommendations. Rather than actively reasoning through the AI’s outputs or explanations, users increasingly accepted them at face value.

Crucially, this effect was not driven by inattentiveness but by reduced analytical engagement. Behavioural measures showed faster response times paired with lower accuracy, indicating shallow processing rather than absent monitoring. Participants in the loafing condition agreed with the AI significantly more often than those who remained analytically engaged, even when the AI’s recommendations were incorrect.

In high-stakes domains, the consequences are particularly severe. In a criminal justice–style decision task, participants who had entered a loafing state were markedly worse at overriding erroneous AI recommendations than those who remained cognitively engaged. Despite being legally and ethically responsible for the final decision, users deferred to the system as if responsibility were shared, resulting in poorer overall outcomes.

Algorithmic loafing illustrates a distinct failure mode of human-in-the-loop systems: humans remain present, attentive, and responsive, yet contribute less reasoning than the system design assumes. Oversight degrades not because users stop monitoring, but because they stop thinking as hard.

Automation-induced complacency

Automation-induced complacency describes a related but temporally distinct phenomenon: the gradual reduction in monitoring and vigilance that occurs when automated systems operate reliably over extended periods. Parasuraman et al. define this as a failure to sustain attention, leading to delayed or missed responses when automation fails.

In a classic simulated engine-monitoring task, Parasuraman et al. demonstrated how quickly this erosion occurs. Participants monitored system gauges and were expected to intervene when failures appeared. For the first 20 minutes, the automation functioned flawlessly, encouraging trust and reducing the perceived need for close attention. When failures were later introduced unpredictably, participants often failed to detect them promptly or respond at all.

When automation performs well, humans disengage to conserve cognitive resources. This strategy becomes dangerous in oversight roles where rare failures carry high consequences.

Mitigating strategies

Identifying automation bias, algorithmic loafing, and automation-induced complacency clarifies why human oversight fails. The harder question is whether these failures can be mitigated. The literature suggests that many intuitive interventions like additional reviewers, procedural training, or increased automation transparency are far less effective than expected. Where mitigation does work is when directly targeting cognitive engagement rather than by adding structural safeguards.

Training for metacognition

Conventional training which tends to focus on procedures like how to operate a system, when to intervene, and what rules to follow, tends to be insufficient. What proves more effective is metacognitive training, that is, explicitly teaching users why cognitive shortcuts occur and how they operate.

Skitka and Mosier show that training which explains the psychological mechanisms behind automation bias significantly reduces commission errors. Users who understand their susceptibility to bias are more likely to slow down, verify unexpected outputs, and resist incorrect recommendations.

Inuwa-Dutse et al. reinforce this distinction. In their study, adding a second human reviewer failed to reduce automation bias. By contrast, training users to recognise algorithmic loafing and over-trust led to more careful verification and higher error detection rates.

Across these studies, adding oversight layers does little to reduce bias; what seems to matter most is the increase of reflective engagement within existing workflows.

Attitudes towards AI

One of the key findings in the Beck et al. paper is that individual attitudes toward AI outperform both expertise and demographic variables as predictors of oversight quality. Participants who were predisposed to trust automation exhibited higher rates of under-correction and acceptance of incorrect suggestions. Those who approached AI with skepticism consistently performed better.

Skeptical users are more likely to question anomalies, notice inconsistencies, and engage in deliberate verification, exactly the behaviours HITL systems depend on.

From a mitigation perspective, this means we should treat individual attitudes towards AI as a potential risk to outcomes, and avoid designing systems that assume uniform performance across users.

Accountability for outcomes

When users know they are responsible for outcomes, they are less likely to disengage cognitively. Clear responsibility increases scrutiny, reduces both omission and commission errors, and counteracts the motivational dynamics underlying algorithmic loafing.

Accountability works not by increasing effort indefinitely, but by preserving a sense of ownership over decisions. Without it, automation invites psychological distancing: errors are attributed to the system rather than to the human operator. HITL designs that obscure responsibility risk accelerating this disengagement.

Implication for UX:
designing for attention

While the above mitigating strategies are useful interventions from a training or governance perspective, what are the implications for UX practioners? Across studies, system and task design emerge as the most reliable levers for mitigation, that is, how might we design interfaces and workflows that sustain attention and invite active judgment.

Several design patterns consistently shape how people engage with AI outputs. Decision-support workflows that slow users down at critical moments reduce uncritical acceptance. Making user actions visible to others introduces social accountability, increasing engagement and care. Cognitive forcing functions such as prompts that require users to explain or justify a decision, push interaction away from confirmation and toward evaluation. Similarly, requiring users to generate corrected values, rather than merely flagging errors, increases cognitive involvement and improves error detection.

Crucially, these patterns work not because they add friction for friction sake, but because they make disengagement harder. They treat attention as a scarce resource that must be actively supported by design. Notably, human checks alone are ineffective unless they promote independent reasoning. Adding reviewers without redesigning the task simply reproduces the same biases in parallel.

Taken together, these strategies point in the same direction: mitigating human–AI cognitive errors depends less on adding oversight and more on designing systems that demand attention, reflection, and ownership from the humans already in the loop.

Conclusion

Human oversight is frequently positioned as a safeguard against AI failure. The evidence reviewed here suggests that this assumption is fragile. Cognitive biases such as automation bias, algorithmic loafing, and automation-induced complacency do not disappear in human–AI systems; they are often amplified by them.

Designing effective HITL systems therefore requires UX practitioners to treat attention, skepticism, and cognitive effort as design concerns. The question is not whether a human is “in the loop,” but whether the system is designed to keep that human meaningfully engaged.

Without that shift, HITL risks becoming a comforting label rather than a reliable safeguard.

References:

Parasuraman, R., Molloy, R., & Singh, I. L. (1993). Performance consequences of automation-induced complacency. International Journal of Aviation Psychology, 3(1), 1–23.

Inuwa-Dutse, I., Toniolo, A., Weller, A., & Bhatt, U. (2023). Algorithmic loafing and mitigation strategies in human-AI teams. Computers in Human Behavior: Artificial Humans, 1(2), Article 100024.

Skitka, L. J., Mosier, K. L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals? International Journal of Aviation Psychology, 10(1), 85–97.

Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). Automation bias: A systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, 19(1), 121–127.

Skitka, L. J., Mosier, K. L., & Burdick, M. (1999). Does automation bias decision-making? International Journal of Human-Computer Studies, 51(5), 991–1006.

Beck, J., Eckman, S., Kern, C., & Kreuter, F. (2025). Bias in the loop: How humans evaluate AI-generated suggestions. Proceedings of the ACM on Human-Computer Interaction.