Tag: Evaluation

All the articles with the tag "Evaluation".

• What Makes a Good Teacher for Knowledge Work Agents

  24 Oct, 2026 · 19 min read

  An overview of agent SFT distillation for research, spreadsheets, documents, and enterprise workflows, with a focus on teacher quality, student compatibility, and completed work.

• How OSWorld Turns Computer Work into an Agent Benchmark

  3 Oct, 2026 · 35 min read

  A technical guide to OSWorld's origins, 13 real tasks, Verified, 2.0 and 2.1, scoring contracts, and the experiments needed to distinguish local progress from complete computer work.

• How Should We Repair Reasoning Traces Before Distillation?

  1 Oct, 2026 · 23 min read

  A map of 26 research papers on reasoning distillation, trajectory correction, and student compatibility: where each method acts, what it costs to implement, and an experiment to test how much of an RL teacher trace to rewrite.

• What Makes an SFT Example Worth Learning? Quality, Novelty, and Learnability

  Updated: 27 Sep, 2026 · 23 min read

  From curriculum learning and DAgger to LIMA, LESS, and on-policy distillation: a source-backed framework for separating SFT data quality, novelty, and learnability, with executable examples and a small negative result.

• Can RFT Learn From Failure? Negative Signals for Agent Fine-Tuning

  20 Sep, 2026 · 16 min read

  A technical guide to learning from failed agent rollouts through step masking, unlikelihood, DPO, SimPO, and step-level credit assignment.

• When Does Model Souping Work for LLMs?

  Updated: 6 Sep, 2026 · 21 min read

  A practical guide to when LLM weight averaging and model merging work, why they fail, how methods such as Task Arithmetic, TIES, DARE, and LoRA merging differ, and how to evaluate a merge before deployment.