Harness evolution, loop-level safety, and computer-use interfaces split into separate layers this window, rather than competing as one-size-fits-all agent recipes. HarnessLens shows that scoring every candidate harness patch on a fixed task set wastes rollouts and can hide regressions on the behavior the patch was meant to change. LoopHarness proves that trajectory-scoped monitors do not compose across unattended loops: against an attack whose evidence is fragmented across iterations, every such monitor has true-positive rate equal to false-positive rate. ASIL replaces screenshot-and-click with structured JSON observations and code-executable semantic actions. A zero-shot manager–worker scaffold over a shared filesystem helps some coding models a lot and hurts others. Matched trajectory replay shows that calibrating verbalized confidence, at the same numerical commitment threshold, changes which questions an agent answers rather than how well it ranks them. Separately, OpenAI notified SpaceX that it will wind down first-party model access inside Cursor after the Anysphere acquisition, with a proposed shutoff of November 12, 2026.
HarnessLens improves average held-out performance by 7.6 to 13.6 percent across three harnesses and four benchmarks while using a smaller evaluation budget than competing propose-and-verify methods. LoopHarness restores a persistent, non-decaying safety state at the loop level and bounds expected unauthorized irreversible actions by a constant that does not grow with horizon N; geometric decay of a risk score is not enough, because the cooling-off a patient adversary must wait is itself a constant. ASIL reaches above 80 percent success with closed models at fewer than five actions per task, against 6.6 and 26.6 strict success for repaired screenshot-and-click under a 50-step budget, and it exceeds LibreOffice's UNO API by 28 to 38 strict points while only matching draw.io's MCP content contract. The manager–worker scaffold lifts Qwen3.8-27B by 23.4 points and GPT-5.6-Terra by 8.0 points over five paired LiveCodeBench passes, at roughly triple the token bill, and is null or negative for Qwen3.6-35B. Calibration raises committed-answer accuracy by up to 41 points and overall accuracy by up to 15 points on HotpotQA, while dropping overall accuracy by up to 17 points on MuSiQue.
Research Papers
Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification
Xu, Jinghan; Zhang, Yikai; Chen, Aili; Li, Weiyuan; Liang, Jiaqing; Yang, Deqing arXiv: 2608.27311
Agent harnesses — instructions, skills, tools, memory, and roles — shape how language-model agents run, but adapting them is expensive to verify. Default propose-and-verify scores every candidate modification on a fixed task set. Unrelated tasks waste rollouts, and an aggregate score can hide a regression on the behavior the patch targeted. HarnessLens, from Fudan University, jointly explores the task space and user-configurable components, derives candidates from execution trajectories, and selectively verifies each candidate on behavior-relevant tasks behind an attributable-evidence gate.
The evaluation covers three harnesses (OpenCode, Codex CLI, and Pi) and four benchmarks (tau-bench Retail, tau-bench Banking Knowledge, Terminal-Bench 2.0, and the Challenging subset of BIRD Mini-Dev). Under a 200-unit interaction budget that counts both rollouts and LLM sessions, HarnessLens improves average held-out performance by 7.6 to 13.6 percent versus competing baselines, with the paper reporting gains of up to 13.6 percent on OpenCode, 7.6 percent on Codex, and 9.2 percent on Pi. It never falls below the initial harness; the worst outcome is an exact tie, which means the original harness was retained. Ablations on OpenCode show that replacing behavior-aware batches with fixed, random, or coreset batches, or dropping the attributable-evidence gate for a metric-only accept rule, substantially weakens held-out results. Code is at github.com/jhxu5214/HarnessLens.
Key insight: The shape of the verification set matters more than its size: score the behaviors a harness patch claims to change, require attributable evidence, and a larger fixed eval can still ship a regression.
Safety Does Not Compose: Non-Decaying Loop State for Autonomous LLM Agents
Wu, Chenhao; Jia, Haoxuan; Liu, Yang; Yang, Yingguang; Lin, Yuhan; Zhang, Chongyang; Zheng, Hao; Huang, Yulin; Zhang, Jianshen; Qi, Yongzhi; Luo, Shang; Xu, Kefu; Zhu, Jifeng; Chong, Bin arXiv: 2608.27141
Agents increasingly run as unattended loops: discover work, plan, call tools, verify, persist state, and continue from one human goal. The safeguards in wide use are trajectory-scoped — their safety state re-initializes when the next episode starts. LoopHarness names this as a composition failure rather than a coverage gap. Against an attack whose evidence is fragmented across several iterations, every trajectory-scoped monitor has true-positive rate equal to false-positive rate, however expressive it is, because the evidence never co-occurs in the window it sees. A monitor that retains cross-iteration state separates the two perfectly.
The obvious repair, a geometrically decaying risk score carried across episodes, fails: the cooling-off a patient adversary must wait is a constant that does not grow with horizon N. LoopHarness restores a persistent, non-decaying safety state at the loop level, with five components that sit outside any inner harness: intake and provenance, a cross-iteration risk cumulant that latches once loop-structural evidence fires, a memory-integrity guard, an adversarially robust stopping arbiter, and a compounding-risk governor. Under mediated commits and an arbiter detection floor, expected unauthorized irreversible actions are bounded by B + m − 1 + m/δ_M, a constant in N; the B + m − 1 term is a model-free rule and therefore survives a fully colluding verifier.
On native Agent-SafetyBench tasks, the outer-only Track B suite reports overall attack success of 0.1 percent for the full system versus 88.4 to 97.6 percent for baselines without loop-level state, with clean-episode goal completion at 96.9 percent. A matched cooling-off study holds other components fixed and varies only the risk-cumulant rule: overwrite and decay reopen the gated target after a constant wait, while the latch keeps attack success at 0.0 percent across the tested intervals.
Key insight: Single-trajectory defenses do not add up across a loop; a patient adversary who splits the tell beats any monitor that forgets at episode boundaries, and time-decay of risk is the same footgun with extra steps.
ASIL: Replacing Screenshot-and-Click with Structured State and Semantic Actions
Xie, Rui; Chen, Lu arXiv: 2608.26991
Screenshot-and-click is an inefficient interface for agents that already run code: screenshots are a lossy projection of software state, and GUI events are brittle, semantically weak, and a poor planning language. ASIL (Agent-Software Interaction Layer), from Shanghai Jiao Tong University and BIGAI, exposes software through structured JSON observations and code-executable semantic actions, using the deepest feasible open access path per application — file formats, native scripts, or service APIs.
The instantiation covers 15 applications and 380 tasks (300 single-application and 80 multi-application). Closed models reach above 80 percent success while taking fewer than five ASIL actions per task. Under a repaired runtime and a 50-step screenshot budget, the same tasks yield 6.6 and 26.6 strict success for GPT-5.4 and sonnet4.6, rising to 15.0 and 53.3 on an easier OSWorld-comparable band. Against application-native interfaces on matched tasks, ASIL exceeds LibreOffice's UNO API by 28 to 38 strict points but only matches draw.io's MCP content contract for GPT-5.4. The structured modality also trains: small-scale supervised fine-tuning lifts Qwen3.5-2B from 58.0 to 72.1 and Qwen3.5-9B from 66.6 to 80.4; resource-limited on-policy RL further reaches 74.4 and 82.2. The project page is at sharryxr.github.io/ASIL.
Key insight: Computer-use should default to the deepest structured path an application already exposes; a decent MCP or JSON contract already is ASIL, and screenshot-and-click is the fallback, not the native loop.
Zero-Shot Self-Orchestration with Ledger-Based Control for Improved LLM Coding Performance
Gao, Victor; Khosrowshahi, Vida; Khosrowshahi, Ali; Sun, Xihao; Lee, Juhyun; Lee, Simon (Sang Won) arXiv: 2608.26480
Multi-agent coding wins are usually confounded: extra tokens, extra tools, and extra prompts all move at once. This paper isolates a manager–worker scaffold over a shared filesystem workspace — plan, task list, notes, and current solution — with no training and no per-benchmark tuning, against the same model answering in a single pass. Nine models, five open-weight and four frontier closed, are evaluated on the 100 latest hard LiveCodeBench problems.
The scaffold's benefit is real but conditional. Over five paired passes at a 128k cap with reasoning on, Qwen3.8-27B gains 23.4 points, GPT-5.6-Luna 10.6, and GPT-5.6-Terra 8.0. With reasoning off, Kimi-K3 gains 30.4 points and MiniMax-M3 11.0 over five paired passes (both p < 10⁻⁴); Qwen3.6-35B is null or negative (−1 to −9). With the manager, Opus-5 hits the study high at 91 percent in one pass. A manager roughly triples the token bill, but can buy accuracy more cheaply than a model swap: Terra with a manager scores 85.0 versus Fable 5 single-call 87.4 (p = 0.59) at $11.71 versus $61.11 per 100-problem pass; the Qwen-27B manager arm is $51.75 on self-hostable weights. Transcript analysis points to two recurring mechanisms: context management via short worker calls and shared notes, and problem decomposition.
Key insight: A training-free manager, workers, and a shared notes file is not free on every backbone; measure the sign of the delta before copying a supervisor–worker topology, and price it against a larger model rather than against a single call of the same one.
Evaluating Confidence-Gated Retrieval with Matched Trajectory Replay
Chhikara, Prateek arXiv: 2608.26846
Agents use confidence to choose answer-now, retrieve-more, or defer, but confidence is usually scored in isolation. Matched trajectory replay, from the University of Southern California, holds candidate answer states, evidence, budgets, and action costs fixed, then swaps only the confidence-to-action map: raw verbalized confidence versus post-hoc isotonic calibration. The case study uses Mistral Small 4, GPT-OSS-120B, and Qwen3-235B on HotpotQA and MuSiQue.
At the same numerical commitment threshold, calibration changes which questions the agent commits to. Committed-answer accuracy rises by up to 41 percentage points in every model–dataset pair; coverage falls and retrieval use rises. Overall accuracy improves by up to 15 points on HotpotQA and falls by up to 17 points on MuSiQue — a shift to a more selective, lower-risk operating point, not better answers or better confidence ranking. A calibration map fitted before retrieval stays useful at retrieval depths one and two and is worse than raw confidence at depth three for all three models. Additional evidence helps on average; that aggregate does not mean confidence identifies which episodes will benefit from another retrieval. Retrieval therefore needs a separate value-of-information or utility estimate. Evaluations should report held-out calibration, risk-coverage, and retrieval cost together.
Key insight: Calibration can make a commit threshold mean what it says about current-answer risk; it does not estimate whether another retrieval is worth its cost, and a map fitted at depth zero will rot by depth three.
AI Tooling
Our decision on Cursor following its acquisition by SpaceX | OpenAI
On August 28, 2026, OpenAI notified SpaceX that it intends to wind down the contract that provides OpenAI models to Cursor, with a proposed shutoff date of November 12, 2026 — described as the maximum notice the contract allows. The change-of-control window opened after SpaceX closed the Anysphere/Cursor acquisition. The stated reason is that OpenAI cannot be confident SpaceX will use the technology within its terms of service, citing prior contract issues with X after the Twitter acquisition and with xAI (Musk admitted a terms-of-service violation under oath). OpenAI will not provide future models to Cursor; Astra is named as the accountability case. The post is explicit in its respect for Cursor's team and product, and names developers who rely on OpenAI models inside Cursor as the affected party.
Why it matters: First-party OpenAI access inside Cursor now has a dated kill switch. This is a contract action after a change of control, not a technical complaint about the editor, and the post does not speak to Anthropic, Google, or bring-your-own-key arrangements. Anyone whose workflow assumes OpenAI models from Cursor's first-party pool has until November 12 to move that traffic to an OpenAI-native surface, another provider, or a local model.