CORTEXA
← Browse
arxivcs.AIcs.CLcs.SE2026-07-09

From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents

Joongho Ahn, Moonsoo Kim

Enterprise large language model (LLM) applications often begin as prototypes whose behavior is carried by prompts and retrieval context. Productization adds requirements for source boundaries, entity routing, answer contracts, and reproducible traces. We present a harness-engineering approach that reconstructs this pattern into a traceable, auditable LLM-agent architecture: deterministic behavior moves into code, manifests, schemas, and validation artifacts around a replaceable composition boundary, while source-backed claims remain the authority for runtime answers. We instantiate it on a public-data slice of five Korean corporate groups (25 listed companies) and evaluate three research questions. (1) The harness preserves its source-grounding, entity-routing, trace, output-hygiene, and recommendation-language contracts across the fixed validation scenarios; a fault-injection control confirms the validators flag deliberately broken contracts. (2) The checks the harness enforces held under model substitution: across three hosted models, they passed on all 270 composition-boundary runs; failures were confined to the model-composed side and were caught and recorded. (3) The code-owned guarantees are load-bearing, not reproducible by prompting alone: holding the model fixed and varying only the enforcement layer, prompt instructions alone let recommendation-language and internal-trace-leakage violations reach the reader, which the harness blocks entirely. A bolt-on external guardrail prevents such violations too but over-refuses, dropping utility to 88/120 where the harness preserves full utility (120/120); in this ablation, only code-owned enforcement preserves both safety and utility. The result is a reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts.

View free PDFSource page

Related papers

arxivcs.AIcs.CLcs.SE2026-07-15

Set-shifting Behavioral Test for Harnessed Agents

Ziwei Ye

What happens to an LLM agent's tool choice when the reliable tool silently changes within an ongoing session? We borrow set-shifting from cognitive psychology to study how well agents adapt to hidden reliability shifts. Our benchmark mounts tool-skill libraries with redundancies,…

View free PDFSource page
arxivcs.CLcs.AIcs.SE2026-07-06

ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents

Harsh Soni

Tool calling is central to modern language model agents, but aggregate benchmark scores often hide where tool use fails. A model that never calls a needed tool and a model that calls the tool but ignores the result can look similar under final task accuracy. We introduce ToolFail…

View free PDFSource page
arxivcs.AIcs.CLcs.SEeess.SY2026-07-14

Do AI Agents Know When a Task Is Simple? Toward Complexity-Aware Reasoning and Execution

Junjie Yin, Xinyu Feng

Large language model (LLM) agents increasingly automate multi-step engineering and informatics workflows, yet they rarely ask how much effort a task actually requires. They often follow a maximum-context-first strategy--re-reading files and dependencies they have already seen--tu…

View free PDFSource page
arxivcs.CLcs.AIcs.HCcs.MAcs.SE2026-07-15

DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environments

Huatao Li, Xinwei Geng, Yuheng Wang, Yutong Li, Runde Yang, Hantao Chen, et al.

LLM-based agents have rapidly improved at operating individual digital environments such as mobile applications, desktop systems, and smart homes. However, real-world user goals often span multiple devices: information may come from a phone, be processed on a desktop, and the res…

View free PDFSource page
arxivcs.SEcs.AIcs.CL2026-07-02

SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces

Jinwei Hu, Yi Dong, Youcheng Sun, Xiaowei Huang

Large Language Model (LLM)-based agents increasingly automate software engineering tasks through reusable skills, natural-language instruction documents that guide planning and execution. Open skill marketplaces enable users to assemble agents by co-activating community-contribut…

View free PDFSource page