LLM Agents

Recommended Posts

• From Tool Calls to Policy Updates: A Reproducible Agent RL Stack

  14 Aug, 2026 · 45 min read

  A reproducible blueprint for Agent RL: environment snapshots, verifier contracts, credit assignment, three-policy semantics, asynchronous generation and training, partial rollouts, and runnable smoke tests.

• A Repair Can Save a Trajectory Without Teaching the Agent

  14 Aug, 2026 · 18 min read

  An eight-seed counterfactual-replay experiment found that high-value one-action repairs reliably rescue failed agent trajectories—but a repair-value-dominant SFT selector did not beat simpler rules and recovered only 25% of matched-RL improvement.

• Beyond 49 ms: Where VM Resume Latency Actually Goes in Agentic RL

  14 Aug, 2026 · 21 min read

  A measurement-first look at fast sandbox resume, speculative tool execution, and action-observation co-speculation—and why time-to-useful-work matters more than boot time.

• Do Verifier Errors Grow Superlinearly with Horizon? A Three-Stage Experiment

  14 Aug, 2026 · 11 min read

  A controlled long-horizon experiment found a clear horizon effect but no preregistered evidence of superlinear verifier error—and exposed why token measurement and false-negative costs matter before scaling RL evaluations.

• Reproducing CompactRL: What Worked, What Failed, and Why We Did Not Scale

  Updated: 24 Jul, 2026 · 16 min read

  An auditable CompactRL reproduction spanning the public algorithm, a 96-step long-horizon simulation, integration with slime, real Qwen actor-critic training, value-function fixes, 17 experimental phases, and the evidence that stopped us from scaling.

• From Long CoT to Agent Swarms: The Documented Evolution of Kimi's Reinforcement Learning

  18 Jul, 2026 · 15 min read

  A source-grounded history of Kimi's reinforcement-learning stack, from Kimi k1.5's long-context outcome RL and partial rollouts to K2's general RL and K2.5's multimodal GRMs and Parallel-Agent RL.

• Training the Critic Without Crashing the Reward: A Practical Guide to Agentic RL

  12 Jul, 2026 · 20 min read

  A practical framework for critic training and credit assignment in long-horizon LLM agents: IQL, pairwise advantage, hindsight and counterfactual critics, privileged information, turn-level MDPs, chain-of-thought monitoring, and reward-crash diagnosis.

• Scaling RL for White-Collar Work: The Environment Foundry

  3 Jul, 2026 · 20 min read

  A practical framework for turning common white-collar workflows into RL environments: spreadsheets, CRM tasks, customer support, web research, dashboards, and other software-mediated work.

• How to Arbitrarily Increase the Difficulty of Agent Evaluation Sets

  28 May, 2026 · 18 min read

  A practical framework for making agent benchmarks harder in a controlled way: treat difficulty as trajectory-graph complexity, not prompt wording. Covers deterministic scoring, capability facets, harness effects, and systematic data generation.

• Improving LLM Internationalization: Bridging the Gap in Tool Use and Agency

  10 Mar, 2026 · 17 min read

  LLMs achieve 57% tool-calling accuracy in English but only 34% across 52 languages — and 6.8% for the worst. This post covers the full playbook for closing the multilingual gap: training-time techniques, agentic architecture patterns, failure mode analysis, and RL-based approaches for i18n.

• Experience-Augmented In-Context Learning: A Training-Free Complement to RL Post-Training

  28 Feb, 2026 · 24 min read

  RL post-training makes models smarter, but it can't cover the infinite long tail of real-world cases. Experience-augmented ICL retrieves successful reasoning traces at inference time, letting agents learn continuously from real usage — no retraining required.

• Tool Selection Optimization for LLM Agents at Scale

  9 Jan, 2026 · 18 min read

  A deep technical dive into tool selection—retrieval strategies, context optimization, learned selection, and the engineering trade-offs that matter when scaling to hundreds of tools.