arxivcs.AIcs.LGcs.SE2026-07-07
AgentLens: Production-Assessed Trajectory Reviews for Coding Agent Evaluation
Andrey Podivilov, Vadim Lomshakov, Sergey Savin, Matvei Startsev, Roman Pozharskiy, Maksim Parshin, et al.
We present AgentLens, a production-assessed benchmark for interactive code agents. Most code-agent benchmarks reduce a run to a single bit -- did the task pass? -- but the people who actually use these agents experience the entire trajectory: how the agent follows instructions, u…