Tag: AI

All the articles with the tag "AI".

• Optimizing Inference for Router Looped Transformers

  Updated: 29 Jun, 2026 · 20 min read

  A research note on serving router looped transformers: why normal KV cache semantics break, what latency data says so far, and how vLLM or SGLang could be adapted with route-template batching and virtual-step KV cache.

• How to Test Pretraining Ideas at Small Scale Before Betting on a Large Model

  29 Jun, 2026 · 25 min read

  A practical guide to validating pretraining improvements with small proxy models, scaling ladders, isoFLOP budgets, loss curves, downstream evals, and rank-correlation checks before committing to an expensive large-model run.

• Why Embeddings Cannot Solve Eval-Set Contamination

  29 Jun, 2026 · 11 min read

  A technical deep dive on why semantic embedding search is useful but insufficient for eval-set decontamination: leakage is about evaluation advantage, not just text similarity.

• Pretraining Contamination: Why Don't Train on the Test Set Became Hard

  29 Jun, 2026 · 14 min read

  A practical introduction to LLM pretraining contamination: why benchmark leakage is not ordinary deduplication, how public evals leak into web-scale corpora, and how layered decontamination pipelines reduce risk.

• How to Arbitrarily Increase the Difficulty of Agent Evaluation Sets

  28 May, 2026 · 18 min read

  A practical framework for making agent benchmarks harder in a controlled way: treat difficulty as trajectory-graph complexity, not prompt wording. Covers deterministic scoring, capability facets, harness effects, and systematic data generation.

• The Mercor Breach: What 4TB of Stolen Data Reveals About How Frontier AI Labs Actually Train Models

  12 Apr, 2026 · 22 min read

  A $10B AI data vendor was breached, exposing 84 Airtable workspaces of training data for OpenAI, Anthropic, Apple, Amazon, and Meta. This post analyzes what the public reporting reveals about each lab's evaluation methodology — rubric design, RLHF pipelines, and quality control — and what it means for the industry.