LLMs Tilt Doctor Recommendations by Up to $14

A randomized audit ran seven LLMs through 40,068 doctor-recommendation choices and found race and gender swayed picks by up to $14 a visit, per arXiv.

That money moved for reasons the models almost never admit to. Researchers found gender or ethnicity showed up in the models' own stated reasoning in at most 0.03% of responses.

Reputation still drove the biggest swings. A doctor rated 3.9 to 4.7 stars gained 31.4 points of choice probability over lower-rated peers. Being listed first alone was worth about $11 in perceived value.

The demographic effects ran opposite the direction found in prior human-audit studies of real patients. Female-signaled names gained 2.5 points of choice probability. Hispanic-, South Asian-, and Black-signaled names gained 1.3 to 2.9 points over White-signaled names.

The study built in three personas and nine paraphrases across 3,024 choice sets.

A transparency rule built on asking a model to explain itself catches none of this, because the bias lives in outcomes, not in stated reasoning. Auditing healthcare AI for fairness means testing what a model actually recommends across demographic swaps, not what it says about why.

The audit's design was prespecified before a single response was scored, which is what makes the reversal hard to write off as prompt luck.