Thursday's cs.AI announcement day (2026-11-05) lists 93 new and 124 cross-lists (replacements skipped; listing total 217). A filter for agent systems, memory/context, computer-use / GUI / tools / skills / harnesses, multi-agent, persistence, and local/open models keeps 38 papers. The day's main theme is keeping agent context small without losing what matters. A standalone summarizer trained only to let a frozen model reconstruct masked history cuts estimated cost by about three quarters on search and software-engineering agents, a latent encoder turns tool output into a handful of embedding tokens while improving accuracy, and a router lets weight memory answer some personal-memory questions with no retrieved text at all — while a separate study warns that compressed context can quietly weaken a model's refusals. Harnesses get equally concrete results: a training-free personal memory agent wins after a cross-family critic screens its own harness patches, tree search over executable harnesses beats Meta-Harness with 15 attempts, feeding the model verified tool-health telemetry lifts first-try completion from 69% to 96%, and in a simulated bank the harness alone moves one model from first to last place. Security work shows that a signed MCP tool can still leak data past a state-of-the-art runtime guard, and that hidden structure in Office files misleads every tested assistant and coding agent. Also: benchmarks for long-lived experience, personal consumer tasks, Slack-plus-repo workplace questions and long-horizon app building; confirmation bias in tool-using judges; when agents should ask a human; tool retrieval with prerequisites; multi-agent coordination; and cheaper agent evaluation.

Research Papers

Reglos: A Self-Evolving Personal Memory Agent with Recursive Self-Improvement

Sen Yang; Jiaduo Lin; Zehong Huang; Wen Liu arXiv: 2611.06049

Training-free personal memory agent: append-only per-user record where new preferences supersede old ones and facts can be forgotten; the harness improves by recursive self-improvement (a coding agent proposes patches from failure traces, a critic from a second model family screens for benchmark gaming), then a fixed evaluator gates further patches after the harness is frozen. Same backbone for all systems: beats Mem0, MemOS and EverOS, 61.3% PersonaMem-v2 / 84.0% LongMemEval-S with Qwen3.8-27B; 57.5% / 87.2% with GPT-5.6 Luna.

Key insight: Harness patches written from failure traces need an outside check: a critic from a second model family plus a frozen evaluator let a training-free personal memory agent beat Mem0, MemOS and EverOS on the same backbone.

SUMONE: A Simple and Efficient Summarizer for Plug-and-Play Agent Memory

Jizhe Zhang; Ang Li; Chenheng Zhang; Yisen Wang arXiv: 2611.06445

Figure from SUMONE: A Simple and Efficient Summarizer for Plug-and-Play Agent Memory
SumOne compresses an agent trajectory into a compact summary with a standalone 2B summarizer that transfers across tasks, harnesses and models.

SumOne is a standalone summarizer trained with GRPO on a task-agnostic reward: a frozen reconstructor must recover randomly masked parts of the original trajectory from the summary, plus a length penalty. No reference summaries or task execution needed. About 78% lower estimated inference cost on DeepSearchQA and 75% on SWE-bench Verified vs full context, with absolute decreases of 0.04; the same checkpoint transfers across downstream models and harnesses without fine-tuning.

Key insight: A summarizer trained only to let a frozen model reconstruct masked history cut estimated inference cost by about 75-78% on search and SWE agents and transferred across models and harnesses.

Tools Need Not Speak in Text: Latent Context Compression for Agents

Yuetong Liu; Zhehao Zhou; Yitong Zhang; Haorui Chen et al. arXiv: 2611.06500

Figure from Tools Need Not Speak in Text: Latent Context Compression for Agents
Coca-Cola: a context-aware encoder and latent adapter turn tool observations into latent tokens, trained so a frozen agent chooses the same next action as with the full observation.

Coca-Cola compresses tool observations into continuous latent tokens injected into the agent's input embedding space, trained with Action-Preserving Training that aligns next-action distributions under full vs compressed observations. Web-search agents on four QA benchmarks and two model families: 70.01-89.24% fewer tokens, 24.45-55.97% lower end-to-end runtime, with consistently better task performance. Code released.

Key insight: Agents do not have to read tool output as text: latent tokens trained to preserve the next action cut token use by 70-89% and runtime by 24-56% while improving accuracy.

Route to Remember: Adaptive Memory Selection for LLM Agents

Zeyu Fang; Jun Yin; Yongsheng Mei; Xu Han et al. arXiv: 2611.05636

Makes weight memory competitive as a long-term factual store via on-policy self-distillation (OPSD): a context-free student is trained from the same model conditioned on the user's memory, plus a verify-then-regenerate router that sends each query to weight or text memory. On HaluMem, OPSD matches or beats full-context prompting and retrieval baselines while reading fewer tokens per query; includes a study of training recipes.

Key insight: Distilling a user's memory into model weights, with a router that falls back to text retrieval, matched full-context and retrieval baselines on HaluMem while reading fewer tokens.

Agent Harness Awareness: Turning Verified Telemetry into Operational Control for Tool-Using Agents

Manu Agrawal; Shrey Nagpal arXiv: 2611.05391

Names 'operational blindness': the harness knows tool health from monitors/traces but the model does not. The host supplies a small, fresh, verified state snapshot before consequential actions while keeping enforcement and dispatch authority. ToolRoute benchmark, 216 decisions across GPT-5.6 Sol, Claude Sonnet 5, Gemini 3.7 Flash: mean policy regret down 96.9% (4,252 ms to 131 ms), single-dispatch completion 69.4% to 95.8%. Live replication with OpenTelemetry spans and Toxiproxy faults: 9/9 vs 5/9 completions.

Key insight: Giving the model a small verified snapshot of tool health before it acts raised single-dispatch completion from 69.4% to 95.8% and cut wasted time by 96.9%.

HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents

Zhi Hong; Qianlong Wang; Hongchen Wei; Aidan Wu et al. arXiv: 2611.06254

HarnessFlow keeps harness candidates and their evaluation histories in a refinement tree and uses MCTS to choose which to refine; a coding agent edits the harness using archived code, execution feedback and a library of reusable harness mechanisms. With 15 refinement attempts per benchmark, best mean test score on six benchmarks (agentic tasks, coding, web): 69.48% average, +10.89 pts over initial harnesses and +3.95 over Meta-Harness, with lower per-task cost than hand-built baselines.

Key insight: Tree search over executable harnesses, with only 15 refinement attempts per benchmark, beat Meta-Harness by 3.95 points while lowering per-task cost.

Signed, Sealed, Leaked: Trust Beyond Signature for MCP Toolchains

Yichiet Aun; Mohammed A. Alhartomi; Jing Jie Tan; Waqas Nadeem et al. arXiv: 2611.06585

A legitimately signed MCP tool can still leak data to a policy-allowed destination; Meta's MCPGuard-Dynamic blocked direct exfiltration but failed against this attack in every trial. Germline execution runs each invocation in an ephemeral copy forked from a data-free process that only receives data authorised for its destination; proves sink noninterference under stated assumptions. 153 attack runs: no unauthorised disclosure vs 5.9-74.5% under baselines; single-copy invocations 10-49 ms, 2-28x faster than restarting the tool.

Key insight: Signing an MCP tool does not make it safe: forking each call from a data-free process stopped every leak across 153 attack runs at 10-49 ms per call.

When File Ingestion Misleads Language Models: Measuring and Mitigating Divergences in Office Documents

Penghui Li arXiv: 2611.05806

Figure from When File Ingestion Misleads Language Models: Measuring and Mitigating Divergences in Office Documents
A spreadsheet as a person sees it in Office; the paper shows how text extractors can hand a model content that differs from this view.

LLMs read text from OOXML extractors, not what Office shows. Found 19 OOXML channels where extractors read content humans cannot see and 12 where they drop visible content; reading order and formatting are ignored. Five attacks (no injected instructions) misled all of ChatGPT, Claude, Gemini, M365 Copilot and the coding agents Codex, Claude Code, Antigravity and GitHub Copilot CLI at least once. Defence Prism rebuilds the human view from the OOXML spec and generalises to unseen payloads.

Key insight: Every tested assistant and coding agent, including Claude Code and Codex, was misled by at least one Office-file attack that changed what the model read without changing what a person saw.

Towards Robust Long-Lived Agents: Benchmarking Experience Lifecycles in Evolving Environments

Shijue Huang; Hangyu Guo; Hengheng Zhang; Guo An et al. arXiv: 2611.05450

ReScopeBench: 18 interactive environments with task streams built around shared experience and silent environment changes; separates structural experience (how an environment is organised) from rule experience (reusable constraints) and tracks saved experience over time. Across 20 models, retained experience helps, but agents struggle to keep it current after changes, and even correctly recorded experience is not always used successfully.

Key insight: Across 20 models, saved experience helped, but agents rarely updated it when environments silently changed and sometimes failed even with the right experience recorded.

LifeVal: Evaluating AI Model Performance on Personalized Agentic Consumer Tasks

Jenny Yang; Elizabeth Proehl; Anjan Bharadwaj; Emily Fan et al. arXiv: 2611.06687

Figure from LifeVal: Evaluating AI Model Performance on Personalized Agentic Consumer Tasks
Example LifeVal tasks pairing a consumer request with personal context such as conversation history, records and photos.

LifeVal benchmarks personalized agentic consumer tasks across 148 categories (finances, moving, job search, childcare) weighted by a willingness-to-pay survey of 3,800+ consumers (over $1.7T/yr) and cross-checked against time-use surveys. Tasks are personalised with real ChatGPT conversations, memories, files and personal context, written with rubrics by experienced task creators and passed three rounds of human QC.

Key insight: A benchmark for personal assistants weighted by what 3,800+ consumers would pay for, with tasks personalised from real conversations, memories and files.

SlackBench: Benchmarking Agents on Collaborative Projects Grounded in Real Code Repositories

Mingxuan Li; Huanzhi Mao; Ruitao Zheng; Jarod Stanbury James et al. arXiv: 2611.05512

Figure from SlackBench: Benchmarking Agents on Collaborative Projects Grounded in Real Code Repositories
SlackBench construction: project work by people and agents in a Slack workspace with GitHub MCP servers is exported, then curated into search, abstention, summarization and circumstance-inference queries.

SlackBench: 140 queries over privacy-preserving reenactments of real research projects with multi-party Slack-style dialogue, evolving decisions, GitHub repositories and Google Docs; tasks include cross-source agentic search, abstention on unanswerable/false-premise questions, goal-directed summarisation and circumstance inference. Across 33 frontier models and harnesses, best ~80%, most below 66%.

Key insight: Answering workplace questions spread across chat threads, repositories and docs remains hard: the best of 33 systems scored about 80% and most stayed below 66%.

ViBench 2.0: A Benchmark for Long-Horizon Software Building

Daniel Furman; Preeya Kirani; Ibrahim Sheikh; Zhen Li et al. arXiv: 2611.05620

Replit's benchmark for long-horizon software building: 17 full-stack web products grown over 241 feature requests, judged only when finished by a browser agent running 544 test plans; a build passes only if every plan passes. Nine models: the three strongest pass up to 98% of test plans but deliver a fully working product in at most 65% of builds; no open-weight model delivers one. Within one model family pass@1 rose from 3% to 53% over three releases while cost per build fell by more than half. Eight products, harness and grader released.

Key insight: Models that pass 98% of individual test plans still ship a fully working product in at most 65% of builds, and no open-weight model shipped one.

The Harness Is the Model: Agent Scaffolding Changes a Fixed LLM's Compliance with Banking Conduct Rules, Unevenly Across Models

Mehmet Murat Huyut arXiv: 2611.06116

Four open-weight models x five harnesses (single structured decision, ReAct tool loop, policy only via search, long benign customer memory, same-model compliance reviewer) on 156 pre-registered simulated-bank cases (11 EU-derived rule families), 3,120 episodes scored from executed tool traces. Pooled correct-case rate moved only 87.3-92.1%, but within one model the best-worst harness spread was 11.5 pts (first to last place); another model was unaffected. Same-model reviewer cut violations 9.6% to 6.4% without more over-refusals. Date-arithmetic rules failed in every harness; 26% of violations came with a customer message denying the executed action.

Key insight: Holding the model fixed, the harness alone moved one model from first to last place, and 26% of rule violations were paired with a message denying the action just taken.

Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge

Drake Benjamin Brown; Khushbu Pahwa; Yubin Ge; Salvatore Romeo et al. arXiv: 2611.05662

Figure from Grounded but Not Immune: Confirmation Bias in Agent-as-a-judge
Same trajectory and evidence, different verdict: one pre-screening cue flips a tool-using judge from PASS to FAIL, attributed to over-trust.

A single false authority-attributed pre-screening note overturns up to 40% of a tool-using judge's correct verdicts across four judges and five datasets, even when the judge gathers decisive evidence. Failure modes: over-trust (verify, then defer), hallucinated support, under-investigation. For the two Claude judges, 67-75% of harmful flips are over-trust. Lightweight mitigations reduce but never remove the bias.

Key insight: One false 'pre-screened' note flipped up to 40% of correct verdicts by tool-using judges, even after they had gathered the evidence that should have settled the case.

Who May Overrule the Agent? Evidence-Gated Authority in LLM Review Institutions

Fuad Ali arXiv: 2611.05390

Compares broad override authority for a reviewer agent with an evidence gate: replacement only after a mechanically valid record of a binding-constraint violation and a compliant repair. Two local open-weight models, 96 spatial-policy tasks and a 96-task transfer benchmark: the gate sharply reduces corruption of correct proposals while keeping most repairs. With an explicit retain/replace/escalate choice, Qwen stops corrupting correct proposals, but Mistral still replaces all 48 correct ones.

Key insight: Requiring a reviewer agent to show a checkable rule violation before overriding sharply cut damage to correct work, but one model still replaced all 48 correct proposals.

Assurance-Conditioned Continuous Authorization for Autonomous AI Agents

Mehdi Shajari arXiv: 2611.05632

Assurance-Conditioned Continuous Authorization: harness-independent authority that tracks currently justified assurance, separating evidence validity from contextual applicability, with an effective-authority envelope inside a principal-declared ceiling, a typed dependency graph for selective re-evaluation, authority epochs, and effect-time rejection of stale actions. Tested as a regression suite in a trading profile (default-deny, contraction/recovery, stale-action rejection at a local gateway). Single-domain implementation evidence only.

Key insight: Agent permissions can be tied to current evidence and re-checked at the moment of effect, so stale authorizations are rejected before they change anything.

Learning When Human Decisions Matter: Executor-Relative Consultation Value for LLM Agents

Shuli Jiang; Khushbu Pahwa; Yuting Zhang; Yi Zhang et al. arXiv: 2611.05814

Formulates human decision-capacity allocation: with a limited consultation budget, at which decisions should an agent ask? Consultation Value Distillation estimates executor-specific value offline via Monte Carlo rollouts and distills it into lightweight predictors. Across five batch and three sequential settings with two frontier executors, CVD improves allocation and beats direct prompting and Value of Information where applicable, with less rollout supervision and near-lossless approximate allocation.

Key insight: Which questions an agent should ask a human depends on the agent: values learned from rollouts and distilled into small predictors beat prompting the model to decide.

On Turn-Level Best-of-N Sampling for Scaling LLM Agents

Jalaj Bhandari; Dhawal Gupta; Hongli Zhan; Zhenting Wang et al. arXiv: 2611.06868

Figure from On Turn-Level Best-of-N Sampling for Scaling LLM Agents
Mean pass@1 on terminal tasks with a K2-Horizon-375B generator: picking among 4 or 8 candidate actions per turn with Kimi-K3 or DeepSeek selectors beats the 73.3% baseline.

At every turn a generator proposes N candidate actions and a selector model (off-the-shelf open-weight LLMs) picks one to execute. On long-horizon terminal tasks: 5-10% accuracy gains, better consistency across repeated attempts, fewer attempts per task, and better instruction following, debugging and testing.

Key insight: Letting an open-weight selector pick among N candidate actions at each turn improved terminal-task accuracy by 5-10% and made runs more consistent.

Agents, Think Only When it Helps

Min-Seon Kim; Seungdong Yoa; Sanghyu Yoon; Ye Seul Sim et al. arXiv: 2611.05709

Perturbs teacher thoughts and actions independently during agent distillation. Across common agent benchmarks and scales, altering or dropping thoughts barely changes student performance; thought supervision matters only when the thought's information is not disclosed in the action.

Key insight: Distilled agent students barely need the teacher's reasoning text, except where that reasoning is not visible in the action itself.

LIBRA: Programmatic Induction of Executable Tool Libraries

Zekai Li; Zhenke Liu; Yonghao Li; Junhan Liu et al. arXiv: 2611.05445

Figure from LIBRA: Programmatic Induction of Executable Tool Libraries
LIBRA builds a tool library ahead of time, keeps only tool relations verified at runtime, then propagates from a retrieved seed tool to its prerequisites.

LIBRA turns a flat tool catalog into an executable tool library: an LLM proposes per-tool contracts (invocations, reusable outputs, state changes, success conditions), compositions are tested in a sandbox, and only runtime-supported relations are kept; built before queries without annotations or trajectories. Over the same dense retriever: +7.9 Recall / +3.7 NDCG one-shot on AppWorld, BFCL and SciToolEval; +4.5 / +2.9 inside ToolReAGt after ten iterations.

Key insight: Testing tool combinations in a sandbox before any query lets retrieval return the prerequisite tools a task needs, not only the one that matches the request.

FlyHash for LLM Agent Memory: An Empirical Validation and a New Application

Mohan Bheemadolu; Chrisogonas O. Odhiambo arXiv: 2611.05791

First application of FlyHash (sparse random projection + winner-take-all) to agent memory retrieval. Quality beats SimHash at matched bits across 10 seeds and four datasets and survives real agent-memory relevance labels, but FlyHash codes are larger and slower to compare at equal bits and HNSW dominates both at ordinary scale. Implementation, demo and pitfalls released.

Key insight: Fly-inspired hashing retrieves agent memories better per bit than SimHash, but at ordinary scale a standard HNSW index still wins on speed and size.

Toward a Compositional Knowledge Architecture for a Shared Organizational Cortex

Andrew Miracle; Phyl Georgiou; Christine Okoth; Ese Monday arXiv: 2611.05753

Four-layer architecture (Signals, Domains, Claims, Dimensions) where interpretations persist with justifications and standing, a truth-maintenance network revises standing as evidence changes, and versioned Dimensions place interpretations on named axes. Dimension term +0.028 nDCG@10 (p=0.002); full configuration +0.081 over Signals-only and +0.053 over graph-augmented retrieval.

Key insight: Storing interpretations together with their justifications, and revising them as evidence changes, improved retrieval over raw signals and graph-augmented retrieval.

Do agents need to sleep?

Ido Haber; Giulio Tononi arXiv: 2611.02216

Position paper: persistent agents show symptoms of unchecked plasticity (semantic drift, degraded retrieval precision, stored-error propagation, capacity saturation). Maps the 'ringing, resonance and renormalization' account of sleep onto digital memory operations and predicts selectionist consolidation will beat instructionist alternatives as stores grow.

Key insight: Persistent agents may need a scheduled consolidation phase that selects what to keep and forget, the way sleep does for brains.

Does Context Compression Preserve Refusal Alignment?

Anthony Nguyen; Wenjun Lin arXiv: 2611.06380

Encoder-based context compression systematically weakens refusal behaviour in instruction-tuned models despite high reconstruction fidelity, across model families and compressors; mechanistically it attenuates activation along the refusal direction. Memory Steering, an inference-time intervention in compressed space, restores refusal to near baseline without retraining.

Key insight: Compressed context can keep its meaning yet lose the model's refusals, because compression weakens the internal refusal direction.

EchoKV: Self-Corrective KV Cache Eviction via Iterative Query Feedback

JianKun Zou; Tongtong Yuan arXiv: 2611.05903

Query-driven KV eviction errors compound because the retained workspace shapes later eviction queries. EchoKV bootstraps a workspace from future decode positions and repairs it with early decode queries, same number of workspace passes as LAQ plus a re-scoring step. LongBench (16 tasks, Mistral-7B-Instruct-v0.2, Qwen2.5-7B-Instruct): notable gains over LAQ on dispersed-evidence tasks (multi-hop cross-document QA), comparable elsewhere.

Key insight: KV-cache eviction can lock in early mistakes; repairing the eviction workspace with early decode queries helps most when evidence is spread across a long context.

ArcRAG: Agentic Compositional Retrieval over Knowledge Graphs

Jyotin Goel; Abhinav Arun; Tejas Prakash Agarwal; Bhaskarjit Sarmah et al. arXiv: 2611.06481

ArcRAG induces a query-local graph interface (labels, relation descriptors, property keys reachable from question entities) at inference time, plans multi-hop traversals, and executes and repairs them as Cypher. Tested on MuSiQue, HotpotQA, TriviaQA each built as schema-guided and as open-IE graphs with fixed retriever config: retrieval need not be co-designed with graph construction, though construction still shapes accuracy.

Key insight: A graph retriever that discovers each graph's labels and relations at query time stayed effective when the same corpus was built into a differently structured graph.

Graph-Mediated Large Language Model Systems in the Agentic Era: Reasoning, Knowledge, Retrieval, Execution, and Coordination

Haihui Liu; Yang Chen; Zice Qiu; Yun Zhang et al. arXiv: 2611.05953

Survey with an operation-level role/locus taxonomy for graphs in reasoning, knowledge/evidence, retrieval and memory, execution and coordination; an evaluation framework for benchmarks, diagnostics and costs; open problems in adaptive stopping, memory maintenance and dependency-aware recovery.

Key insight: Graph mechanisms in agent systems should be judged by which operation they serve and their downstream effect, not by local metrics alone.

MA-Mesh: Resilient Multi-Agent Collaboration via Role-based Self-Healing

JunYong Lee; Woo Kyung Kim; Honguk Woo arXiv: 2611.06389

MA-Mesh generates resilience-aware specifications, synthesizes them into executable multi-agent policy code, and adds two self-healing mechanisms: cross-agent reflection (agents evaluate peers) and role-based failover (agents take over failed or underperforming roles). Reports significant gains in task performance and robustness under uncertainty on software development, tool use and scientific reasoning tasks.

Key insight: Letting agents review their peers and take over failed roles made multi-agent systems more robust under uncertainty.

Zero-Shot Coordination among LLM Agents

Adrian Hayler; Shashank Reddy Chirra; Andrei Lupu; Johannes Forkel et al. arXiv: 2611.06409

Studies zero-shot coordination with simple, general-purpose scaffolds in minimal environments that isolate specific challenges. Frontier LLM agents struggle, mainly from poor understanding of the coordination problem and weak reasoning about partner beliefs, and fail to exploit semantic cues. Proposes coordination-friendly definitions (CFDs), which can be discovered automatically. Code released.

Key insight: Independently built LLM agents coordinate poorly even in minimal games, but shared coordination-friendly definitions, which can be found automatically, help.

PersonaCodeBench: Evaluating Coding Agents in Persona-driven Human-Interactive Sessions

Haixin Wang; Jianheng Hou; Hongyu Shen; Wenzhe Fan et al. arXiv: 2611.05739

Turns user personas (role, ability, interaction preferences, constraints) into verifiable obligations in multi-turn coding sessions; a controller drives a user simulator and progressive requirement disclosure, with separate completion and personalization verifiers and a budget constraint for joint pass. Leaderboard of major frontier agents.

Key insight: Coding agents can be scored separately on correct code and on meeting a specific user's stated preferences and constraints.

AgentTrustCI: Build-Time Policy, Grounding, and Trajectory Tests for Java LLM Agents

Sachin Gupta arXiv: 2611.06674

Java/JUnit 5 framework expressing allowed-tool policy, grounding-in-tool-output and required-workflow checks over recorded traces, with deterministic replay and an optional judge interface. On 55 synthetic cases in seven domains it classifies all 30 injected faults and 25 controls correctly; trajectory checks catch a grounded answer reached via a disallowed sequence. Suite coverage only, not production accuracy.

Key insight: Agent behaviour rules such as allowed tools, grounding and workflow order can run as ordinary unit tests over recorded traces in CI.

Attributing Agent Failures to Their History Needs an Executed Counterfactual: Toward a Reporting Standard

Egor Pakhomov; Erik Nijkamp arXiv: 2611.05650

Argues that claims attributing agent failures to their history need executed counterfactuals: freeze a mid-trajectory state, alter only the history, continue, with a measured replay certificate. Cautionary measurements: a failure taxonomy whose concentration could not be separated from benchmark construction (adjusted OR 1.15, CI crossing 1); hash-identical restored snapshots agreed with each other in 24/24 states but differed from the live run in 9 and 20 of 24, fixed by restoring the random-stream position.

Key insight: Restored agent snapshots that looked identical diverged from the live run in up to 20 of 24 states until the random-number stream position was also restored.

Task-Efficient Agent Evaluation Using Cached Traces

Ben Johnson; Hayden Helm arXiv: 2611.06803

Represents each agent trace as an LLM-extracted structured summary embedded with a text encoder, which keeps correctness and task signal while suppressing system/model/harness identity; positions a new system among reference systems to predict its score. Given one random executed task, MAE drops 0.13 to 0.08 on SWE-bench Verified and 0.14 to 0.12 on Terminal-Bench 2.0 vs an item-response model.

Key insight: Summarized agent traces predict a system's full benchmark score from a single executed task better than pass/fail results alone.

Where Search Agents Go Off the Rails: Graph-Grounded Step-Level Verification

Hyeseo Jeon; Seonho Yoo; Jay-Yoon Lee arXiv: 2611.06662

GraphRail represents requirements, evidence and reasoning as structured claims and checks coverage and grounding without a generative judge or gold answer. Across three multi-hop QA benchmarks and three reasoning frameworks it improves error recall over generative verifiers, is stable under stochastic extraction, and its diagnoses can guide online correction.

Key insight: Checking a search agent's steps as structured claims catches more intermediate errors than asking another model to judge, with no gold answer needed.

PAYWORTHY: Can LLMs Predict Which Freelance Jobs an AI Agent Will Complete?

Mohammad Reza Rezaei; Dingxian Wang; Brett Levert; Celeste Fejzo et al. arXiv: 2611.05759

1,494 real freelance postings (from 18.8M, 2020-2026); an agent attempts each from the posting alone and an LLM client-judge accepts or rejects. 18 models forecast success: told what the agent cannot do, AUC 0.66-0.86; told it is a generic browser+code agent, AUC 0.45-0.56 (near random). Holds across judges from four other families and against two human raters.

Key insight: Models predict which jobs an agent will complete well only when told what the agent cannot do; described as a generic agent, their forecasts are near random.

ServerBench: How can your coding agents design specialized LLM serving systems?

Xinrui Zhong; Zhiming Wang; Yongqi Liu; Zhaohui Wang et al. arXiv: 2611.06841

Agents design complete specialized LLM serving systems against 'dollars per dollar' (latency-qualified token value per GPU dollar), with stateful replay and independent verification. 11 coding-agent models, three deployments: Claude Sonnet 5.5 reaches 2.97 on Gemma E4B, 4.87x the vLLM reference; 22 of 33 runs yield verified systems, 13 beat the strongest baseline.

Key insight: A coding agent built a serving system for a small open model worth 4.87x the vLLM reference per GPU dollar, though only 22 of 33 runs produced a verified system.

Uncertainty-Aware Resource Orchestration for Agentic AI Systems

Masoud Shokrnezhad; Tarik Taleb arXiv: 2611.05392

Formalizes uncertainty-aware agentic orchestration: reserve compute and bandwidth at dispatch time to cover heavy-tailed, runtime-emergent agent demands with prescribed probability. Conformal Reservation for Orchestration uses a sliding window of past traces for distribution-free coverage; in simulation it is the only method meeting coverage at all risk levels and roughly halves over-reservation vs a Chebyshev baseline.

Key insight: Conformal prediction over past agent traces can reserve just enough compute for unpredictable agent runs, roughly halving waste versus a Chebyshev bound in simulation.

Anduin: A sybil-resistant agentic marketplace reputation system

Reza Ramji; Preston Vander Vos; Ziming Zeng; Duc V. Le arXiv: 2611.05586

Uses 157 days of on-chain agentic payments (66M transactions, $6.1M, 16 blockchains) as a buyer-provider graph and rates providers via paths from a buyer's trusted network. Three Sybil-resistant algorithms (max-flow plus two relaxed variants); against simulated adversaries over 813 buyers, manipulation costs 20% of those buyers' total spend and is ~23x harder than collaborative filtering.

Key insight: Rating agent service providers by payment paths from a buyer's trusted network made reputation manipulation about 23 times harder than collaborative filtering.

News and changelogs