Tag: Post-Training

All the articles with the tag "Post-Training".

• Reproducing CompactRL: What Worked, What Failed, and Why We Did Not Scale

  Updated: 24 Jul, 2026 · 16 min read

  An auditable CompactRL reproduction spanning the public algorithm, a 96-step long-horizon simulation, integration with slime, real Qwen actor-critic training, value-function fixes, 17 experimental phases, and the evidence that stopped us from scaling.

• From Long CoT to Agent Swarms: The Documented Evolution of Kimi's Reinforcement Learning

  18 Jul, 2026 · 15 min read

  A source-grounded history of Kimi's reinforcement-learning stack, from Kimi k1.5's long-context outcome RL and partial rollouts to K2's general RL and K2.5's multimodal GRMs and Parallel-Agent RL.

• Training the Critic Without Crashing the Reward: A Practical Guide to Agentic RL

  12 Jul, 2026 · 20 min read

  A practical framework for critic training and credit assignment in long-horizon LLM agents: IQL, pairwise advantage, hindsight and counterfactual critics, privileged information, turn-level MDPs, chain-of-thought monitoring, and reward-crash diagnosis.

• From GRPO Outcome Rewards to Token-Level Advantage

  8 Jul, 2026 · 20 min read

  A practical framework for turning GRPO-style sequence rewards into token-level advantages, including GAE-style estimators, credit assignment routes, and multi-reward training design.

• The Mercor Breach: What 4TB of Stolen Data Reveals About How Frontier AI Labs Actually Train Models

  12 Apr, 2026 · 22 min read

  A $10B AI data vendor was breached, exposing 84 Airtable workspaces of training data for OpenAI, Anthropic, Apple, Amazon, and Meta. This post analyzes what the public reporting reveals about each lab's evaluation methodology — rubric design, RLHF pipelines, and quality control — and what it means for the industry.

• Instruction Following: What Models Get Wrong and How to Fix It with Better Post-Training Data

  4 Mar, 2026 · 36 min read

  LLMs can write poetry and solve math, but ask them to 'respond in exactly 3 bullet points using only lowercase' and they stumble. This post dissects the taxonomy of instruction-following failures and provides a practical playbook for building post-training data that actually fixes them.