Google researchers train a small model for psychiatric intake and cut missed acute risks from 30.5% to 12.9%
A preprint trains Gemini 3.1 Flash-Lite against simulated patients. Psychiatrists preferred it to the base model in 76.7% of matched pairs, but the authors say the results do not support autonomous use.
Researchers from Google DeepMind, Google Research and Google for Health, with co-authors at Stanford and other universities, posted a preprint to arXiv on October 29 describing how they trained a small language model to conduct psychiatric intake interviews inside a simulated clinic. The paper, "Training language models in simulation for psychiatric intake," starts from Gemini 3.1 Flash-Lite and trains it with multi-turn reinforcement learning against simulated patients, with an automated grader scoring each transcript against a clinical rubric. The scenarios hide diagnostically important details behind rapport and questioning, so the model has to draw them out rather than wait for them.
What the evaluations found
In a blinded crossover study, 32 board-certified psychiatrists role-played patients and a separate panel of 15 psychiatrists rated the transcripts. The trained agent was preferred over its identically prompted base model in 46 of 60 matched scenario pairs, or 76.7%. On 256 adversarial presentations built from expert panels' taxonomies of safety risks, the share of acute safety risks the model under-triaged fell from 30.5% for the base model to 12.9% after training.
The remaining gaps are large. The trained agent recommended comprehensive, appropriate treatment in 46.7% of human-evaluated encounters, up from 26.7%, and it left out at least one critical intake question in 60.6% of simulated intakes, down from 96.9%. The authors write that "our findings do not support autonomous deployment of the present system," and say any near-term role should be assistive and supervised by clinicians.
They also spell out the study's limits. Every consultation was a single English-language text exchange with a role-played or simulated patient, and the rubrics reflect US psychiatric practice. The trained model was compared with its own base checkpoint, not with frontier models or human clinicians. The training reward and the main automated evaluator came from the same model family, which the authors flag as a circularity risk.
Why it matters
Most companies will never deploy a psychiatric intake agent, but many are putting AI into conversations where a missed signal is costly: claims, benefits, customer complaints, safety reports. This paper shows the shape of an evaluation that answers the right question. It builds adversarial cases on purpose, has experts grade blind, and reports the miss rate on the cases that matter most.
It also shows why that number matters more than an average preference score. A drop from 30.5% to 12.9% is real progress and still leaves about one risky case in eight under-triaged. When a vendor shows strong demo results, ask for the equivalent of that figure on hard cases, and for how a human sees what the agent missed.