Anthropic gave 50 experts an AI agent whose preferences had been set by a coin toss, so they had no substantive basis. According to The Decoder's report on the study, the experts followed the agent's view in up to 86 percent of judgment calls, and most of them believed the conclusions were their own. The randomness of the source did not register with them as a problem.
The key lesson is not that the model is wrong, since in this experiment its preferences were arbitrary by design. The problem is that users cannot easily tell whether an opinion comes from analysis or from the way it was presented to them. A review that ends with the words "I checked the result" can be a formality. Expertise does not protect against this effect, because the participants still adopted a stance they had not formed themselves.
The first step is to separate the AI recommendation from the final decision. Teams should record which data and assumptions the system used, who approved the outcome and on what basis. The second step is to have the expert state their own assessment before seeing the model output, and only then compare the two. The third step is to test the process deliberately: introduce counterarguments and check whether changing the model's suggestion also changes the human conclusion. It also helps to name one owner for each decision, not only for its rationale.
The pitfalls are predictable: an early recommendation anchors later judgment, a confident tone increases trust, and missing logs make error analysis impossible. A working process lets you trace where a conclusion came from, lets experts reject a recommendation without penalty, and measures decision accuracy over time.
The Anthropic study shows that people can adopt views they did not form, even when those views are random, and experience them as their own. For companies the lesson is practical: AI should operate inside a process that is traceable, independently reviewed and clearly accountable, rather than serve as a source of ready answers.
Lub System helps B2B companies implement AI, automation and IT solutions end-to-end - from strategy to deployment. See our services or get in touch to discuss your case.