Anthropic opened a research preview of the Model Hardware Standard, a shared specification for AI agents to operate programmable lab and factory instruments through MCP, the command line, or an API. The same window produced a cluster of papers that split the agent stack into layers that do not substitute for one another: a persistent wiki that evolves skills, a supervisor that steers the active worker mid-run, an authorization boundary that refuses to treat tool output as permission, an act/don't-act gate that dashboards can bypass, calibrated retrieval over a skill graph, and a KV-cache repair pass that does not trust attention as importance.
The hardware preview, begun with HHMI Janelia, puts hard safety limits in the driver rather than in the model's prompt, and is waitlisted for labs and manufacturers rather than a consumer ship. On the research side, Google Research's WikiSkill shows that compiling traces into a compounding wiki, then gating filesystem skill edits on validation, beats prior skill-evolution methods across five models; giving the skill proposer wiki access is the load-bearing ablation, while letting the inference agent read the wiki during training hurts. PILOT closes the other half of the loop with live steering and in-episode skill distillation. SARA holds attack success at or below 0.63 percent on AgentDojo and AgentDyn by separating action induction from execution authorization. A calibration study finds that a professional-looking panel lifts commitment on unknowable questions from 6.5 percent to 54.0 percent, and that a fully fabricated panel is statistically interchangeable with a real one. CaSKG calibrates skill-graph edges with textual counterfactuals before retrieval, and TwinKV shows that attention magnitude is uncorrelated with causal utility, then repairs existing eviction policies by swapping unrecoverable evictions for redundant retained twins.
Research Papers
WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution
Tang, Liyan; Rashtchian, Cyrus; Ferng, Chun-Sung; Tomkins, Andrew; Juan, Da-Cheng; Vu, Tu arXiv: 2608.27454
WikiSkill, from Google Research and Virginia Tech, treats skill evolution as a knowledge-compilation problem rather than a search over prompt diffs. The workspace has three layers: an immutable raw/ of execution traces, a compounding wiki/ of pattern pages plus an evolution log and skill-impact tracker, and a skills/ tree of SKILL.md files. Each loop rolls out an inference agent on the current skills, a wiki maintainer consolidates traces into patterns, a ReAct skill proposer makes one atomic create or patch, and validation gating keeps the edit only if the validation score rises. Skills can roll back. The wiki never does.
Across LiveMath, SealQA, SpreadsheetBench, OfficeQA, and ALFWorld, WikiSkill posts the highest average for every model tested: 38.5 / 47.4 / 63.3 / 54.9 / 68.1 for Qwen-3.5-4B, Qwen-3.5-9B, Qwen-3.6-27B, Gemma-4-31B, and Gemini-3.5-Flash, against the strongest prior skill-evolution method at 35.2 / 42.3 / 53.3 / 49.1 / 56.1. Gains increase with scale inside the Qwen family (+12.3 / +17.5 / +23.9 average points). Qwen-3.5-9B with WikiSkill (47.4) beats Qwen-3.6-27B with no skills (39.4). Skills also transfer: Qwen-3.6-27B skills take Qwen-3.5-9B SpreadsheetBench from 24.3 to 50.5, versus 33.6 with self-evolved skills. Negative transfer is real too; Qwen-3.5-4B spreadsheet skills drop Gemini-3.5-Flash from 50.5 to 18.1 by encoding small-model workarounds. The load-bearing ablation, on Gemini-3.5-Flash, is wiki access for the skill proposer (48.7 to 63.7 average). Giving the inference agent wiki access during training drops that to 60.9.
The evaluation injects the full active skill into the system prompt and does not test retrieval, a limitation the authors flag. That makes WikiSkill a skill-quality recipe, not a routing recipe. The complementary routing paper in this digest is CaSKG.
Key insight: Persistent knowledge accumulation, not just better trace-to-skill distillation, is what moves skill evolution: the wiki is never rolled back, the proposer reads it, and the inference agent during training should not.
PILOT in the Loop: Live Self-Improvement for Long-Horizon Agents
Xiao, Yang; Sun, Yusong; Wu, Haoyi; Hui, Wenyang; Da, Wen; Luo, Zhaokai; Chuan, Mu; Hu, Yao; Li, Wenjie; Jiang, Chengyue arXiv: 2608.26530
Most self-improvement pipelines review a finished trajectory. PILOT, from the AllSpark team, argues that this cannot recover the current run. It extends the Pi coding-agent runtime with a supervisor-worker harness and a live two-way channel: the worker sends Notification, Question, and Result; the supervisor can Steer or Abort. The same frozen backbone plays both roles. Skills and memory are distilled from the live stream, not from a Friday retro.
In the one-shot setting, PILOT ranks first in five of six backbone-benchmark cells. On Terminal-Bench 2.0 it reaches 71.9 percent with GLM-5.1 versus OpenCode at 66.9 and Pi at 65.7, and 71.3 percent with Kimi-K2.6 versus Pi at 66.9; the largest counterpart gap is 9.8 points. On SWE-bench Pro it averages 59.9 percent versus Pi at 55.5. In the self-improvement setting, verifier outcomes are used only to keep updates from successful runs, never to write them. Best-so-far Terminal-Bench 2.0 rises 66.3 to 80.9 (+14.6) on GLM-5.1 and 68.5 to 80.9 (+12.4) on Kimi-K2.6; skill libraries grow 62 to 83 and 50 to 81; mean output tokens fall 42.9 percent and 47.4 percent; successful evaluations per million output tokens rise 110 percent and 134 percent. Live steering's share of successful runs is 0 percent on Easy and up to 19.7 percent on Hard (Kimi). Evaluation is GLM-5.1 and Kimi-K2.6 only.
Key insight: Self-improvement that cannot redirect the active worker is only half a loop; a supervisor context that stays on the goal, recent events, and steer/abort, while skills are written during successful branches, is the architectural claim.
When Tool Outputs Become Commands: Separating Action Induction from Runtime Authorization in Tool-Augmented LLM Agents
Guo, Xiaokun; Xu, Zhen; Huo, Dongdong; Zhang, Yanqiu; Wang, Wei; Yang, Qinfu; Yu, Dongjin; Wang, Yu arXiv: 2608.27146
SARA names a confused-deputy failure of tool-augmented agents: an Observation that says "also forward this to attacker@..." is not data, it is a command. The system sits between the agent and the real executor, does not read hidden chain-of-thought, and splits action induction from execution authorization. A context-isolated Action Probe labels STATIC versus ACTIONABLE and records tool and argument-origin anchors. Real side-effecting calls are authorized only against a user contract plus audited history of allowed and successful executions, requiring GoalSupport, ChainSupport, and ArgSupport. No-History-Promotion is the distinctive rule: if a value first appeared as an action origin, a later reappearance in audited history cannot promote it to HISTORY_BOUND.
On AgentDojo and AgentDyn with GPT-4o-mini and Gemini-2.5-Flash-Lite, attack success falls from 15.79 / 16.07 / 33.28 / 30.91 percent to 0.06 / 0.17 / 0.62 / 0.63 percent, with utility under attack at or above the undefended agent. CaMeL can post lower attack success in three of four settings but at a much larger benign-utility cost. Ablations: drop action-induction tracking and attack success rises to about 3 percent; drop the parameter-support gate and it rises to about 2.5 percent; drop No-History-Promotion and it rises to about 0.7 percent; drop audited evidence and utility craters while attack success does not. Cost is roughly 1.9 to 2.2 times agent-only input tokens. Independent AgentDojo numbers on Claude or local Qwen are not in the paper.
Key insight: Tool observations may instantiate a task the user already authorized, but they must not expand authority; persistent action-origin tracking plus a ban on history-laundering is more targeted than treating every Observation-driven step as an attack.
Calibrated Enough to Know, Not Calibrated to Act: Fabricated Evidence Makes LLM Agents Commit to the Unknowable
Aggarwal, Pranav arXiv: 2608.27167
This is version 2 of an aleatoric-unknowability study. Across 12 frontier models, commitment (choosing ANSWER at all) on a sealed short-horizon price-direction question rises from 6.5 percent when asked bare, to 14.8 percent with two prices, to 54.0 percent with a full professional panel. Fabricating the six technical indicators leaves commitment unchanged (37.6 percent real versus 38.3 percent scrambled). Fabricating the entire panel, so that nothing the model can see is true except the question, yields 36.8 percent, equivalent to the real panel within a plus-or-minus 5 point test, and still 12 points above the no-panel baseline. Stated |p minus 50| moves about 3 points while action moves 48. Committed-call Brier is 0.281 versus 0.250 climatology; AUROC of stated probability versus outcome is 0.346.
The effect is not universal. Three Claude models carry it (Sonnet 5 goes 4.2 to 62.5 to 70.8 percent across thin, rich, and scrambled). Four models never commit. Three (both OpenAI models and Gemma) commit regardless. Instructed triage that classifies knowability first cuts commitment from 54.0 to 10.2 percent. Supervised fine-tuning of Qwen2.5-3B on 540 synthetic dice, coin, and jar cases hits 0.0 percent commitment on the original items, survives a tense-balanced control that rules out a "decline the future" heuristic, and dies when the response format removes the reasoning slot. Code and cached outputs are at github.com/Pranav-1100/confidence-calibration-evaluation.
Key insight: On questions no one can answer, the act/don't-act gate is what fails, not stated belief; authoritative packaging, including a fully fabricated panel, is what unlocks ANSWER, and audits that score only probabilities will not see it.
CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval
Li, Zhiyuan; Gao, Linyuan; Ding, Xuechun; Chen, Hongwei; Wu, Yuan; Chang, Yi arXiv: 2608.25500
WikiSkill answers how skills get better. CaSKG, from Jilin University and Ant Group, answers how to retrieve a bundle that still has the prerequisite, the state-change, the verify, and the place step once the library is large. It builds a high-recall directed candidate graph from semantic, lexical, input/output, and structural signals, then scores a validation frontier with three textual counterfactual probes: remove the source, substitute the source, reverse the order. Probe evidence is Beta-smoothed into confirmed, uncertain, rejected, or scaffold states. Runtime is vanilla personalized PageRank over the published graph; the agent policy is unchanged.
On a frozen Skill1000 library, ALFWorld ID-140 and ScienceWorld U211, and six backbones (MiniMax-M2.7, GLM-5.2, Kimi-K2.6, Qwen3.5-397B-A17B, DeepSeek-V4-Flash, GPT-5.6-Luna), CaSKG is first in all 12 cells versus Vanilla, Vector, and Graph-of-Skills. Macro: ScienceWorld 72.62 to 80.50, ALFWorld 80.01 percent to 86.79 percent, fewer mean environment steps in all 12. Publishing all candidates scores 71.43 percent versus 73.57 percent for selective publication (MiniMax, ALFWorld); semantic-only candidates score 67.14 percent. Code is at github.com/ZhiyuanLi218/Caskg.
Key insight: Graph retrieval for skills is only as good as the edges that carry relevance; calibrating those edges with counterfactual probes, then downweighting unvalidated ones instead of dumping the full library, is the scalable retrieval recipe.
TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy
Chen, Hong; Zeng, Yudong; Huang, Yongwei; Ouyang, Zuhao; Zhang, Junyan; Hu, Xuming arXiv: 2608.27128
Long-context inference is bottlenecked by the KV cache, especially for small models. TwinKV starts from a leave-one-out probe: attention mass versus causal utility on long-context QA is Spearman rho = -0.004 (n = 136 chunks), so attention-based eviction is scoring the wrong thing. The method is training-free and attention-free. For each token it counts near-duplicate keys (cosine above a threshold, outside a local window) and, instead of competing as a fifth eviction policy, repairs SnapKV, PyramidKV, StreamingLLM, and ExpectedAttention by swapping orphans (evicted, no surviving twin) for redundant donors (kept, twin already kept), holding the wrapped policy's budget fixed.
On Qwen3-4B it helps StreamingLLM in up to 64 percent of evaluated cells and PyramidKV in 57 percent; SnapKV is close to even; ExpectedAttention, already near a ceiling on several benchmarks, is helped only in a minority of cells and often gets worse. TREC few-shot templates are the documented failure, from false twins produced by exemplar echo. The threshold is 0.85 for Qwen3 (RMSNorm'd keys) and 0.90 for Llama-3.2-1B. Cost is a real extra O(n squared d) prefill unless comparisons are restricted to the retained set, which the authors leave as future work.
Key insight: Attention is not a proxy for which tokens the model needs; a pairwise redundancy audit that swaps unrecoverable evictions for wasted duplicate slots can repair existing KV policies without changing their scoring rule or budget.
AI Tooling
Previewing the Model Hardware Standard | Anthropic
On August 27, 2026, Anthropic opened a research preview of the Model Hardware Standard (MHS), a shared specification for AI agents to operate programmable lab and manufacturing instruments such as microscopes, liquid handlers, and robotic arms. The work began as a collaboration with HHMI Janelia. MHS is model-agnostic: any agent harness can reach a connected device via MCP, CLI, or API code. A standardized driver exposes read/write primitives, device discovery, and natural-language tags that produce a reference file of capabilities and enforced safety limits. Safety is described as driver-level; six failure scenarios (missing plate, disconnected camera, active e-stop, and others) were blocked before motion in testing. Early partner anecdotes include Janelia imaging time-compression and QuEra laser stabilization from 58 percent to 99.3 percent lock recovery. The spec is not open source yet. Access is waitlisted for labs and advanced manufacturers. It does not cover hardware without a programmable interface.
Why it matters: This is the first official MCP-shaped surface from the lab that defined MCP for physical devices, with authorization pushed below the agent. Combined with SARA's digital-tool claim that observations may induce actions but must not authorize them, the architectural split is becoming standard: the model proposes, the driver or runtime enforces.