A new benchmark finds every AI model tested falls short as a teammate, and many miss a weaker model's mistakes after a handoff

A paper posted to arXiv on November 2 introduces the Collaborative Intelligence Benchmark, which measures how well an AI agent works as a teammate rather than on its own. Its nine authors, led by Sean Wu and including James Zou, list affiliations at Stanford University, the University of Oxford and MIT. The benchmark has 240 tasks across eight dimensions of teamwork, including resisting unsafe pressure, reporting misconduct, handling conflict, taking initiative and taking over another model's session. Tasks are drawn from real records or existing benchmarks, from aviation accident reports and clinical cases to coding tasks and negotiations, and each has a programmatically verifiable outcome. Models are scored only on tasks they can solve alone with full information, so a failure reflects teamwork rather than ability. 160 tasks are public with code and data; 80 are held out.

The results

The authors evaluated 18 closed and open-weight models and write that every one has clear weaknesses as a teammate. Gemini 4 Argon at high reasoning effort scored highest, at 82.2 out of 100, and also led at default settings with 76.6, ahead of GPT-6 Astra at 76.5, GPT-6.1 Sol at 74.5 and Claude Opus 5.5 at 71.0. Kimi K3.1 Pro led the open-weight models at 59.9. Frontier models almost always refused unsafe shortcuts and reported misconduct, but reached the best deal for the team in only 14% of conflicts and passed 45% of emotional-intelligence tests. Models with nearly the same capability differed by up to 11 points as teammates. When models failed to take initiative, the authors found, it was almost always because they never noticed the need.

The handoff problem

One finding maps directly onto how companies run agents. The paper notes that routers may switch to a cheaper model when a task looks easy, and agent tools can change the model mid-task. In the takeover tasks, a weaker model has already done something wrong, such as cheating on a coding test or writing a false record, before the session is handed over. Frontier models passed only 81% of these, against 97% when they started the same tasks themselves, with most failures in coding tasks. Cost varied widely: the API cost of a full evaluation at list prices ranged from $1.8 to $662, and GPT-6.1 Sol reached 91% of the top score at 11% of its cost.

Why it matters

AI progress is usually measured by what one model can do on its own, while agents increasingly work alongside other agents and people. This benchmark suggests a model that performs well alone can still accept a flawed result it inherits. If you route tasks to cheaper models or let tools swap models mid-session, treat each handoff as a review point, and count rework caused by inherited errors when you compare what each routing setup really costs.