Posts

All the articles I've posted.

• Training the Critic Without Crashing the Reward: A Practical Guide to Agentic RL

  12 Jul, 2026 · 20 min read

  A practical framework for critic training and credit assignment in long-horizon LLM agents: IQL, pairwise advantage, hindsight and counterfactual critics, privileged information, turn-level MDPs, chain-of-thought monitoring, and reward-crash diagnosis.

• From GRPO Outcome Rewards to Token-Level Advantage

  8 Jul, 2026 · 20 min read

  A practical framework for turning GRPO-style sequence rewards into token-level advantages, including GAE-style estimators, credit assignment routes, and multi-reward training design.

• A Looped Transformer Router Shows Its First Replicated Gain

  6 Jul, 2026 · 11 min read

  A small-budget BPE language-model experiment where a sparse late-final-loop token-feedback router becomes the first route-looped Transformer candidate to beat matched fixed-loop baselines across several controlled checks.

• Scaling RL for White-Collar Work: The Environment Foundry

  3 Jul, 2026 · 20 min read

  A practical framework for turning common white-collar workflows into RL environments: spreadsheets, CRM tasks, customer support, web research, dashboards, and other software-mediated work.

• When a Looped Transformer Router Almost Works

  1 Jul, 2026 · 12 min read

  A controlled small-scale language-composition experiment comparing fixed and routed looped Transformers, showing why the first sequence-level router was competitive but collapsed toward a weak exit policy.

• Optimizing Inference for Router Looped Transformers

  Updated: 29 Jun, 2026 · 20 min read

  A research note on serving router looped transformers: why normal KV cache semantics break, what latency data says so far, and how vLLM or SGLang could be adapted with route-template batching and virtual-step KV cache.