CORTEXA
← Browse
arxivcs.CLcs.AIcs.LG2026-07-22

Frontier Financial Judgement: Can agents tell what might move a stock?

Joshua Harris

We introduce Frontier Financial Judgement, a challenging new benchmark developed in collaboration with professional equity analysts to assess agents' ability to replicate expert human judgements. Rapidly identifying new information, evaluating its implications and determining its valuation impact is one of the most time-consuming and challenging aspects of real-world equity coverage. This is becoming ever more difficult and important as AI rapidly increases the quantity of new information to process. The strongest agent we evaluate on Frontier Financial Judgement matches all expert labels in only 52.4% of cases. We also find significant divergence in estimated false-positive rates among frontier agents, ranging from ~1% for GPT-5.6 Sol to ~32% for Claude Sonnet 4.6. To construct the benchmark and make it representative of real-world settings, we combine human-designed and labelled synthetic articles with live news articles and historical documents, creating 656 items for assessment. The resulting task requires agents to distinguish genuinely new, valuation-relevant financial information from stale, immaterial or misleading news under realistic conditions. We find substantial trade-offs among agent accuracy, cost, false positives and reliability that continue to hinder the reliable deployment of news-flow filtering in practice.

View free PDFSource page

Related papers

arxivcs.AIcs.CLcs.LG2026-07-14

Self-Improvements in Modern Agentic Systems: A Survey

Zhe Ren, Yimeng Chen, Dandan Guo, Guowei Rong, Tonghui Li, R. B. Xiong, et al.

Self-improving autonomous agents are moving from research prototypes to deployed systems. The primary goal is controllable evolution, or adaptation, from experience with minimal or even no human input. This survey frames modern self-improving agents as adaptive systems that conve…

View free PDFSource page
arxivcs.CLcs.AIcs.HCcs.LG2026-07-10

A small language model detects behavioural faithfulness gaps that frontier judges and human raters miss

Kwan Soo Shin, In Seok Kang, Yunkyung Min, Munho Lee

Whether a language model behaves as it claims is a judgement on which independent human raters cannot agree (Fleiss kappa = 0.074). We show that a small, purpose-built instrument does better. A linear read-out of the frozen representation of a from-scratch 146-million-parameter a…

View free PDFSource page
arxivcs.AIcs.CLcs.LG2026-07-22

DocOps: A Verifiable Benchmark for Autonomous Agents in Complex Document Operations

Jiazhen Jiang, Boxi Cao, Lingyong Yan, Yaojie Lu, Hongyu Lin, Shuaiqiang Wang, et al.

As autonomous agents rapidly evolve, their ability to reliably manipulate ubiquitous digital documents has become critical for enabling general-purpose AI assistants and automating complex workspace workflows. In this paper, we introduce DocOps, a deterministically verifiable eva…

View free PDFSource page
arxivcs.LGcs.AIcs.CL2026-07-01

Mapping the Evaluation Frontier: An Empirical Survey of the Bias-Reliability Tradeoff Across Eleven Evaluator-Agent Conditions

Zewen Liu

The bias-reliability tradeoff conjectures that LLM evaluation systems are constrained in (gamma, H, CV) space, where evaluator coupling (gamma), strategy diversity (H), and small-sample measurement reliability (CV(N)) cannot be simultaneously optimized at fixed sample size N. Pri…

View free PDFSource page
arxivcs.AIcs.CLcs.LGcs.MA2026-07-11

Ontology-Amplified Distillation and Contextuality Auditing for Sovereign Enterprise Language Models: A Combined Proof-of-Mechanism and Negative-Results Method Study

Thanh Luong Tuan

Regulated financial institutions operating under data-residency rules need tenant-owned language models that can run inside the institution's perimeter. This paper combines two related FAOS studies into one mechanism-and-control article. First, it reports a reduced-power proof-of…

View free PDFSource page