Article: Designing for Human–AI Collaboration
Reviewed: November 2026 · Method version 2 · How these reviews work
Each factual claim in the article was checked against the source it cites, then challenged by reviewers whose job was to find the strongest evidence against it: replication attempts, meta-analyses, later papers that cite the original work, and research from other fields that study the same mechanism. A separate reviewer weighed both sides, and a script re-checked every quoted source. Where a claim was overstated or misattributed, the article was corrected. The research below supports the article as it now reads.
Algorithm aversion is abandoning a better forecaster after watching it make a mistake.
The definition of algorithm aversion matches the cited paper; a 2017 experiment replicated the drop in reliance on automated advice after a mistake.
Automation bias is following an aid when it is wrong, both by missing problems it failed to flag and by acting on its bad advice.
The definition matches Parasuraman and Manzey’s review.
A 2010 review by Parasuraman and Manzey found automation bias in novices and experts alike, and in individuals and teams.
Later work in law and medicine found automation bias across expertise levels, as the 2010 review reported.
A 2010 review by Raja Parasuraman and Dietrich Manzey found it in novices and experts alike, in individuals and teams, and concluded that training or instructions cannot prevent it. Later reviews found they can reduce it, so treat training as a help, not a safeguard.
Parasuraman and Manzey concluded training cannot prevent automation bias; a 2012 healthcare review found training and accountability reduce it.
For a designer, the useful point is that neither failure comes down to accuracy alone. Both depend on how the person forms a picture of the system’s reliability, and the interface is where much of that picture gets formed.
The framing draws on trust-in-automation research (Lee and See 2004), now credited.
Before choosing whether to tie their bonus to their own forecasts (in one study, another participant’s) or to a statistical model built from the same data, participants saw the model’s track record, the human’s, both, or neither.
Clarified that in most of the 2015 studies the human forecaster was the participant.
The model won every study: human forecasts carried 15-29% more error in the MBA task and 90-97% more in the airline task.
Figures match the paper.
People who had watched the model perform were less likely to bet on it.
Core finding confirmed; later ‘algorithm appreciation’ work studied people before they saw errors, so it does not undercut this result.
Of the participants who saw both forecasters, 610 of 741 watched the model beat the human, and that group was among the least likely to choose the model.
Count matches the paper.
Watching the human forecaster err, who in most studies was the participant, did not significantly reduce people’s willingness to bet on that human.
Clarified that the ‘human’ in most studies was the participant.
Pull quote: ‘It seems that the errors that we tolerate in humans become less tolerable when machines make them.’ (Dietvorst, Simmons and Massey 2015)
Pull quote is verbatim.
Every visible AI feature shows its errors sooner or later, so the finding is relevant to most of them, though later studies show its strength varies with the task, the user and whether people are comparing the system with their own judgment.
Later reviews show algorithm aversion varies with task, user and comparison point.
When participants could modify an imperfect algorithm’s forecasts, they were considerably more likely to use it, even when the adjustments allowed were severely restricted, and they did better than people who relied on their own forecasts. People given that small amount of control were more satisfied and more willing to use the algorithm again. A 2023 replication confirmed this only in part, so treat it as a strong lead rather than a settled rule.
A 2023 replication partly confirmed the 2018 ‘modifiable algorithm’ result.
In a 2021 study, Zana Buçinca, Maja Barbara Malaya and Krzysztof Gajos found that people shown AI suggestions with explanations still followed the AI when it was wrong, as earlier studies had also found. Three designs that forced a moment of thought reduced that overreliance: asking people to decide before seeing the AI’s answer, making the answer available only on request, or showing it after a delay.
Buçinca et al. did observe overreliance in both of their explanation conditions, but their experiment had no AI-without-explanation condition, so the comparative point that explanations fail to reduce overreliance rests on earlier studies (Bansal et al. 2021; the Schemmer et al. 2022 meta-analysis) and on a 2023 clinical trial in JAMA. Later work finds explanations do reduce overreliance in some conditions: when decisions are easy (2024), when explanations are cheap to engage with relative to the task (2023), when explanations are partial (2025), and in a 2026 radiology study where chain-of-thought explanations beat bare answers. The honest summary is that explanations alone are an unreliable guard against overreliance, not that they never help.
Three designs that forced a moment of thought helped: asking people to decide before seeing the AI’s answer, making the answer available only on request, or showing it after a delay. Those designs reduced overreliance compared with simple explanations, though none removed it.
Clarified that cognitive forcing reduced, but did not eliminate, overreliance.
The cognitive forcing designs were the least liked, and they helped people who enjoy effortful thinking (high Need for Cognition) more than others.
Matches the paper.
How much people were allowed to change the forecast made relatively little difference to whether they chose the algorithm; being able to change it at all did. The desire was for some control, not a lot of it.
Matches the 2018 paper’s Study 2.
In 2019 Amershi and colleagues at Microsoft distilled more than 150 design recommendations into 18 guidelines for human-AI interaction, tested by 49 design practitioners against 20 AI products.
Figures match the paper.
Three that the study found important but still not widely implemented were making clear how well the system does what it does, explaining why it did something, and giving users global controls.
Wording adjusted to credit the authors’ analysis rather than practitioners’ direct judgment.
Amershi’s first two guidelines ask products to make clear what the system can do and how often it may be wrong, and another guideline asks products to support efficient correction when the system is wrong.
Guideline content matches the paper.
A short line at the point of use (“Good at first drafts. Check names and numbers.”) turns that mistake into something the person was told to expect, and in one Microsoft study, setting expectations this way made people more accepting of an imperfect assistant. Say it before the first output, not on a help page. Then keep it specific: a marker on uncertain answers, or a note on what the system couldn’t check, tells people where to look, which a general disclaimer does not.
The expectation-setting advice now credits Kocielnik, Amershi and Bennett (2019).
Buçinca’s results, from a lab task with everyday participants, say this reduces overreliance and that people will like it less. A study of radiologists found a cost as well: those who made their own call first agreed less with the AI even when it was right.
A clinical imaging study (Fogliato et al. 2022) found judgment-first lowers agreement with both right and wrong AI advice.
These edits were made on November 12, 2026 after the review.
Claim reviewed: The review concluded that training or instructions cannot prevent automation bias.
Now reads: A 2010 review by Raja Parasuraman and Dietrich Manzey found it in novices and experts alike, in individuals and teams, and concluded that training or instructions cannot prevent it. Later reviews found they can reduce it, so treat training as a help, not a safeguard.
Why: The article reports the 2010 review’s conclusion accurately: it says automation bias cannot be prevented by training or instructions. But a later systematic review of 74 healthcare studies lists training and accountability as factors that reduce it, so ‘cannot prevent’ should not be read as ‘does not help’. Now cites Goddard, Roudsari & Wyatt, Automation bias: a systematic review of frequency, effect mediators, and mitigators, JAMIA 19(1), 2012.
Claim reviewed: Neither failure is mainly about accuracy; both are about how the person forms a picture of the system’s reliability, and the interface is where that picture gets formed.
Now reads: For a designer, the useful point is that neither failure comes down to accuracy alone. Both depend on how the person forms a picture of the system’s reliability, and the interface is where much of that picture gets formed.
Why: This is the article’s own framing, not a cited finding. Research on trust in automation does say reliance follows people’s picture of a system’s reliability, shaped partly by the display, but accuracy and error patterns still matter, so ‘neither failure is mainly about accuracy’ overstates it. Now cites Lee & See, Trust in Automation: Designing for Appropriate Reliance, Human Factors 46(1), 2004.
Claim reviewed: Dietvorst, Simmons and Massey ran five studies in which participants forecast MBA applicants’ program performance and, in two studies, US states’ ranks on departing airline passengers, then chose whether to tie their bonus to a human’s forecasts or a statistical model, after seeing the model’s track record, the human’s, both, or neither.
Now reads: Before choosing whether to tie their bonus to their own forecasts (in one study, another participant’s) or to a statistical model built from the same data, participants saw the model’s track record, the human’s, both, or neither.
Why: The study design is described correctly, except for one detail that matters: in Studies 1 to 3 the human forecaster was the participant, and only Study 4 used another person. Much of the aversion is a preference for one’s own judgment over the model’s.
Claim reviewed: Watching the human err did not significantly reduce people’s willingness to bet on the human.
Now reads: Watching the human forecaster err, who in most studies was the participant, did not significantly reduce people’s willingness to bet on that human.
Why: The paper says seeing the human err did not significantly reduce choosing the human, and a 2017 experiment found the same asymmetry. But in most of the studies ‘the human’ was the participant, so this mostly shows people sticking with their own forecasts, which the article does not say.
Claim reviewed: Every visible AI feature shows its errors sooner or later, so the algorithm-aversion finding applies to nearly all of them.
Now reads: Every visible AI feature shows its errors sooner or later, so the finding is relevant to most of them, though later studies show its strength varies with the task, the user and whether people are comparing the system with their own judgment.
Why: The drop in reliance after an algorithm errs has been replicated, so it is relevant to most visible AI features. But reviews of 60 to 80 studies show its strength depends on the task (subjective tasks are worse), the user, and whether people compare the system with their own judgment, so ‘applies to nearly all of them’ is too flat. Now cites Mahmud, Islam, Ahmed & Smolander, What influences algorithmic decision-making? A systematic literature review on algorithm aversion, Technological Forecasting and Social Change 175, 2022.
Claim reviewed: When participants could modify an imperfect algorithm’s forecasts they were considerably more likely to use it, even when the adjustments allowed were severely restricted, performed better as a result, and were more satisfied and more willing to use the algorithm again.
Now reads: When participants could modify an imperfect algorithm’s forecasts, they were considerably more likely to use it, even when the adjustments allowed were severely restricted, and they did better than people who relied on their own forecasts. People given that small amount of control were more satisfied and more willing to use the algorithm again. A 2023 replication confirmed this only in part, so treat it as a strong lead rather than a settled rule.
Why: The 2018 paper says exactly this, including higher satisfaction. A 2023 replication confirmed the effect only in part and raised reproducibility concerns, and ‘performed better’ means better than people using their own forecasts, not better than the unmodified algorithm. Now cites Cheng & Chouldechova, Overcoming Algorithm Aversion: A Comparison between Process and Outcome Control, CHI 2023.
Claim reviewed: Buçinca, Malaya and Gajos (2021) found that adding explanations to AI suggestions did not stop people over-relying on them.
Now reads: In a 2021 study, Zana Buçinca, Maja Barbara Malaya and Krzysztof Gajos found that people shown AI suggestions with explanations still followed the AI when it was wrong, as earlier studies had also found. Three designs that forced a moment of thought reduced that overreliance: asking people to decide before seeing the AI’s answer, making the answer available only on request, or showing it after a delay.
Why: The full text confirms that both of the study’s explanation conditions showed people following the AI when it was wrong, so the sentence’s attribution holds as written: Buçinca’s own participants over-relied despite explanations. Two caveats remain. The study never compared explanations against no explanations, so the comparative phrasing ‘adding explanations did not stop’ leans on earlier work the paper summarizes, and later studies show explanations can reduce overreliance when the task is easy, the explanation is cheap to check, or the explanation is partial, so the point should not read as settled. Now cites Bansal et al., Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance, CHI 2021.
Claim reviewed: Three designs that forced a moment of thought (deciding before seeing the AI’s answer, answer on request only, answer after a delay) did stop overreliance, reducing it compared with simple explanations.
Now reads: Three designs that forced a moment of thought helped: asking people to decide before seeing the AI’s answer, making the answer available only on request, or showing it after a delay. Those designs reduced overreliance compared with simple explanations, though none removed it.
Why: The three designs are described correctly and the paper reports they significantly reduced overreliance compared with simple explanations. But the paper says they did not eliminate it, and the article’s ‘did’ implies they stopped it.
Claim reviewed: Three guidelines that practitioners judged relevant but not widely implemented were making clear how well the system does what it does, explaining why it did something, and giving users global controls.
Now reads: Three that the study found important but still not widely implemented were making clear how well the system does what it does, explaining why it did something, and giving users global controls.
Why: The paper does name Guidelines 2, 11 and 17 as important but not widely implemented. That label is the authors’ reading of where practitioners found the guidelines applied or violated, not a judgment the practitioners made themselves.
Claim reviewed: Aversion grows when the first mistake arrives as a surprise; telling people at the point of use what to expect, and marking specific uncertain answers, reduces it in a way a general disclaimer does not.
Now reads: A short line at the point of use (“Good at first drafts. Check names and numbers.”) turns that mistake into something the person was told to expect, and in one Microsoft study, setting expectations this way made people more accepting of an imperfect assistant. Say it before the first output, not on a help page. Then keep it specific: a marker on uncertain answers, or a note on what the system couldn’t check, tells people where to look, which a general disclaimer does not.
Why: This advice is uncited but has support: a Microsoft study found that setting expectations before use made people more accepting of an imperfect AI assistant. Evidence that specific uncertainty markers work better than a general disclaimer is thinner; confidence scores helped people calibrate trust but did not by themselves improve decisions, so ‘never does’ overstates it. Now cites Kocielnik, Amershi & Bennett, Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems, CHI 2019.
Claim reviewed: For high-stakes decisions, asking for the person’s call before revealing the AI’s answer reduces overreliance, and people will like it less.
Now reads: Buçinca’s results, from a lab task with everyday participants, say this reduces overreliance and that people will like it less. A study of radiologists found a cost as well: those who made their own call first agreed less with the AI even when it was right.
Why: Buçinca’s ‘decide first’ condition did reduce overreliance and was liked less, but it was tested with lay participants on a food task, not money or health decisions. A study of radiologists found judgment-first also made experts agree less with the AI when it was right, a cost the article does not mention. Now cites Fogliato et al., Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging, FAccT 2022.