<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"><channel><title>ML &amp; AI in Action</title><description>Explore the Practical Side of ML &amp; AI: Real-World Applications and Impact.</description><link>https://ajing.github.io/</link><item><title>What Makes a Good Teacher for Knowledge Work Agents</title><link>https://ajing.github.io/posts/2026-10-24-distilling-knowledge-work-agents/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-10-24-distilling-knowledge-work-agents/</guid><description>An overview of agent SFT distillation for research, spreadsheets, documents, and enterprise workflows, with a focus on teacher quality, student compatibility, and completed work.</description><pubDate>Sun, 25 Oct 2026 05:42:59 GMT</pubDate></item><item><title>How OSWorld Turns Computer Work into an Agent Benchmark</title><link>https://ajing.github.io/posts/2026-10-03-osworld-task-design-long-horizon-workflows/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-10-03-osworld-task-design-long-horizon-workflows/</guid><description>A technical guide to OSWorld&apos;s origins, 13 real tasks, Verified, 2.0 and 2.1, scoring contracts, and the experiments needed to distinguish local progress from complete computer work.</description><pubDate>Sun, 04 Oct 2026 01:00:00 GMT</pubDate></item><item><title>How Should We Repair Reasoning Traces Before Distillation?</title><link>https://ajing.github.io/posts/2026-10-01-repairing-reasoning-traces-before-distillation/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-10-01-repairing-reasoning-traces-before-distillation/</guid><description>A map of 26 research papers on reasoning distillation, trajectory correction, and student compatibility: where each method acts, what it costs to implement, and an experiment to test how much of an RL teacher trace to rewrite.</description><pubDate>Thu, 01 Oct 2026 23:00:00 GMT</pubDate></item><item><title>What Makes an SFT Example Worth Learning? Quality, Novelty, and Learnability</title><link>https://ajing.github.io/posts/2026-09-27-sft-quality-novelty-learnability/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-09-27-sft-quality-novelty-learnability/</guid><description>From curriculum learning and DAgger to LIMA, LESS, and on-policy distillation: a source-backed framework for separating SFT data quality, novelty, and learnability, with executable examples and a small negative result.</description><pubDate>Mon, 28 Sep 2026 00:32:00 GMT</pubDate></item><item><title>Can RFT Learn From Failure? Negative Signals for Agent Fine-Tuning</title><link>https://ajing.github.io/posts/2026-09-20-negative-signals-agent-rft/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-09-20-negative-signals-agent-rft/</guid><description>A technical guide to learning from failed agent rollouts through step masking, unlikelihood, DPO, SimPO, and step-level credit assignment.</description><pubDate>Mon, 21 Sep 2026 01:00:00 GMT</pubDate></item><item><title>Improving LLM Internationalization: Bridging the Gap in Tool Use and Agency</title><link>https://ajing.github.io/posts/2026-03-11-improving-llm-i18n-tool-use-agency/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-03-11-improving-llm-i18n-tool-use-agency/</guid><description>A practical multilingual agent playbook, updated with task-specific evidence from MASSIVE-Agents, GAIA-v2-LILT, and SEATauBench, with explicit limits on what transfers to current frontier models.</description><pubDate>Sun, 06 Sep 2026 07:00:00 GMT</pubDate></item><item><title>From GRPO Outcome Rewards to Token-Level Advantage</title><link>https://ajing.github.io/posts/2026-07-09-grpo-token-level-advantage/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-07-09-grpo-token-level-advantage/</guid><description>A practical framework for turning GRPO-style sequence rewards into token-level advantages, including GAE-style estimators, credit assignment routes, and multi-reward training design.</description><pubDate>Sun, 06 Sep 2026 07:00:00 GMT</pubDate></item><item><title>When Does Model Souping Work for LLMs?</title><link>https://ajing.github.io/posts/2026-09-02-when-does-model-souping-work-for-llms/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-09-02-when-does-model-souping-work-for-llms/</guid><description>A practical guide to when LLM weight averaging and model merging work, why they fail, how methods such as Task Arithmetic, TIES, DARE, and LoRA merging differ, and how to evaluate a merge before deployment.</description><pubDate>Sun, 06 Sep 2026 07:00:00 GMT</pubDate></item><item><title>Do LLM Agents Work Equally Well Across Languages?</title><link>https://ajing.github.io/posts/2026-09-05-multilingual-agent-language-gaps/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-09-05-multilingual-agent-language-gaps/</guid><description>Agent performance differs across languages, tasks, and localization settings. A visual review of 30 languages and 16 evidence sources identifies where Arabic, Thai, Japanese, and Hindi need targeted evaluation.</description><pubDate>Sun, 06 Sep 2026 03:30:00 GMT</pubDate></item><item><title>What Makes an Agent Swarm Work? Simple Rules, Information Boundaries, and the Value of Verification</title><link>https://ajing.github.io/posts/2026-09-05-agent-swarm-organization-simple-rules/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-09-05-agent-swarm-organization-simple-rules/</guid><description>A theory-and-experiment study of agent swarm organization: operational roles, marginal-contribution incentives, selective information sharing, and why verification remains the limiting assumption.</description><pubDate>Sat, 05 Sep 2026 19:00:00 GMT</pubDate></item><item><title>Do Verifier Errors Grow Superlinearly with Horizon? A Six-Stage Experiment</title><link>https://ajing.github.io/posts/2026-08-14-verifier-error-horizon-scaling/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-08-14-verifier-error-horizon-scaling/</guid><description>A preregistered 224-artifact study confirms a 66% hybrid-verifier improvement, does not support the universal superlinear headline, and finds that task structure can reverse the apparent horizon effect.</description><pubDate>Mon, 17 Aug 2026 07:00:00 GMT</pubDate></item><item><title>From Tool Calls to Policy Updates: A Reproducible Agent RL Stack</title><link>https://ajing.github.io/posts/2026-08-14-agent-rl-reproducible-closed-loop/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-08-14-agent-rl-reproducible-closed-loop/</guid><description>A reproducible blueprint for Agent RL: environment snapshots, verifier contracts, credit assignment, three-policy semantics, asynchronous generation and training, partial rollouts, and runnable smoke tests.</description><pubDate>Sat, 15 Aug 2026 02:35:03 GMT</pubDate></item><item><title>A Repair Can Save a Trajectory Without Teaching the Agent</title><link>https://ajing.github.io/posts/2026-08-14-counterfactual-trajectory-repair-sft-vs-rl/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-08-14-counterfactual-trajectory-repair-sft-vs-rl/</guid><description>An eight-seed counterfactual-replay experiment found that high-value one-action repairs reliably rescue failed agent trajectories—but a repair-value-dominant SFT selector did not beat simpler rules and recovered only 25% of matched-RL improvement.</description><pubDate>Fri, 14 Aug 2026 23:30:00 GMT</pubDate></item><item><title>Beyond 49 ms: Where VM Resume Latency Actually Goes in Agentic RL</title><link>https://ajing.github.io/posts/2026-08-14-beyond-49ms-agentic-rl/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-08-14-beyond-49ms-agentic-rl/</guid><description>A measurement-first look at fast sandbox resume, speculative tool execution, and action-observation co-speculation—and why time-to-useful-work matters more than boot time.</description><pubDate>Fri, 14 Aug 2026 18:00:00 GMT</pubDate></item><item><title>Reproducing CompactRL: What Worked, What Failed, and Why We Did Not Scale</title><link>https://ajing.github.io/posts/2026-07-17-reproducing-compactionrl-glm-5-2-slime/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-07-17-reproducing-compactionrl-glm-5-2-slime/</guid><description>An auditable CompactRL reproduction spanning the public algorithm, a 96-step long-horizon simulation, integration with slime, real Qwen actor-critic training, value-function fixes, 17 experimental phases, and the evidence that stopped us from scaling.</description><pubDate>Sat, 25 Jul 2026 05:30:00 GMT</pubDate></item><item><title>From Long CoT to Agent Swarms: The Documented Evolution of Kimi&apos;s Reinforcement Learning</title><link>https://ajing.github.io/posts/2026-07-18-from-long-cot-to-agent-swarms-kimi-rl/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-07-18-from-long-cot-to-agent-swarms-kimi-rl/</guid><description>A source-grounded history of Kimi&apos;s reinforcement-learning stack, from Kimi k1.5&apos;s long-context outcome RL and partial rollouts to K2&apos;s general RL and K2.5&apos;s multimodal GRMs and Parallel-Agent RL.</description><pubDate>Sat, 18 Jul 2026 17:30:00 GMT</pubDate></item><item><title>Training the Critic Without Crashing the Reward: A Practical Guide to Agentic RL</title><link>https://ajing.github.io/posts/2026-07-12-training-critics-without-crashing-reward-agentic-rl/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-07-12-training-critics-without-crashing-reward-agentic-rl/</guid><description>A practical framework for critic training and credit assignment in long-horizon LLM agents: IQL, pairwise advantage, hindsight and counterfactual critics, privileged information, turn-level MDPs, chain-of-thought monitoring, and reward-crash diagnosis.</description><pubDate>Sun, 12 Jul 2026 23:00:00 GMT</pubDate></item><item><title>A Looped Transformer Router Shows Its First Replicated Gain</title><link>https://ajing.github.io/posts/2026-07-06-looped-transformer-router-finally-starts-to-work/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-07-06-looped-transformer-router-finally-starts-to-work/</guid><description>A small-budget BPE language-model experiment where a sparse late-final-loop token-feedback router becomes the first route-looped Transformer candidate to beat matched fixed-loop baselines across several controlled checks.</description><pubDate>Mon, 06 Jul 2026 18:00:00 GMT</pubDate></item><item><title>Scaling RL for White-Collar Work: The Environment Foundry</title><link>https://ajing.github.io/posts/2026-07-03-scaling-rl-for-white-collar-work-environments/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-07-03-scaling-rl-for-white-collar-work-environments/</guid><description>A practical framework for turning common white-collar workflows into RL environments: spreadsheets, CRM tasks, customer support, web research, dashboards, and other software-mediated work.</description><pubDate>Fri, 03 Jul 2026 18:00:00 GMT</pubDate></item><item><title>When a Looped Transformer Router Almost Works</title><link>https://ajing.github.io/posts/2026-07-01-when-a-looped-transformer-router-almost-works/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-07-01-when-a-looped-transformer-router-almost-works/</guid><description>A controlled small-scale language-composition experiment comparing fixed and routed looped Transformers, showing why the first sequence-level router was competitive but collapsed toward a weak exit policy.</description><pubDate>Wed, 01 Jul 2026 18:30:00 GMT</pubDate></item><item><title>Optimizing Inference for Router Looped Transformers</title><link>https://ajing.github.io/posts/2026-06-29-optimizing-looped-transformer-inference/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-06-29-optimizing-looped-transformer-inference/</guid><description>A research note on serving router looped transformers: why normal KV cache semantics break, what latency data says so far, and how vLLM or SGLang could be adapted with route-template batching and virtual-step KV cache.</description><pubDate>Tue, 30 Jun 2026 01:10:00 GMT</pubDate></item><item><title>How to Test Pretraining Ideas at Small Scale Before Betting on a Large Model</title><link>https://ajing.github.io/posts/2026-06-29-testing-pretraining-ideas-across-model-scales/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-06-29-testing-pretraining-ideas-across-model-scales/</guid><description>A practical guide to validating pretraining improvements with small proxy models, scaling ladders, isoFLOP budgets, loss curves, downstream evals, and rank-correlation checks before committing to an expensive large-model run.</description><pubDate>Mon, 29 Jun 2026 10:00:00 GMT</pubDate></item><item><title>Why Embeddings Cannot Solve Eval-Set Contamination</title><link>https://ajing.github.io/posts/2026-06-29-why-embeddings-cannot-solve-eval-contamination/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-06-29-why-embeddings-cannot-solve-eval-contamination/</guid><description>A technical deep dive on why semantic embedding search is useful but insufficient for eval-set decontamination: leakage is about evaluation advantage, not just text similarity.</description><pubDate>Mon, 29 Jun 2026 09:00:00 GMT</pubDate></item><item><title>Pretraining Contamination: Why Don&apos;t Train on the Test Set Became Hard</title><link>https://ajing.github.io/posts/2026-06-29-pretraining-contamination/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-06-29-pretraining-contamination/</guid><description>A practical introduction to LLM pretraining contamination: why benchmark leakage is not ordinary deduplication, how public evals leak into web-scale corpora, and how layered decontamination pipelines reduce risk.</description><pubDate>Mon, 29 Jun 2026 08:00:00 GMT</pubDate></item><item><title>How to Arbitrarily Increase the Difficulty of Agent Evaluation Sets</title><link>https://ajing.github.io/posts/2026-05-28-agent-eval-difficulty-trajectory-constraints/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-05-28-agent-eval-difficulty-trajectory-constraints/</guid><description>A practical framework for making agent benchmarks harder in a controlled way: treat difficulty as trajectory-graph complexity, not prompt wording. Covers deterministic scoring, capability facets, harness effects, and systematic data generation.</description><pubDate>Thu, 28 May 2026 08:00:00 GMT</pubDate></item><item><title>The Mercor Breach: What 4TB of Stolen Data Reveals About How Frontier AI Labs Actually Train Models</title><link>https://ajing.github.io/posts/2026-04-13-mercor-breach-what-leaked-about-ai-training/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-04-13-mercor-breach-what-leaked-about-ai-training/</guid><description>A $10B AI data vendor was breached, exposing 84 Airtable workspaces of training data for OpenAI, Anthropic, Apple, Amazon, and Meta. This post analyzes what the public reporting reveals about each lab&apos;s evaluation methodology — rubric design, RLHF pipelines, and quality control — and what it means for the industry.</description><pubDate>Mon, 13 Apr 2026 00:00:00 GMT</pubDate></item><item><title>The Unverifiable Reward Problem: The Real Frontier of RL for LLMs</title><link>https://ajing.github.io/posts/2026-03-08-unverifiable-rewards-rl-frontier/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-03-08-unverifiable-rewards-rl-frontier/</guid><description>Deep research on tasks with unverifiable rewards in RL — the key bottleneck for scaling RL beyond math and code. Covers JEPO, NRT, RLNVR, self-play methods, GenRM, Constitutional AI, reward hacking mitigation, and more.</description><pubDate>Sun, 08 Mar 2026 00:00:00 GMT</pubDate></item><item><title>Instruction Following: What Models Get Wrong and How to Fix It with Better Post-Training Data</title><link>https://ajing.github.io/posts/2026-03-05-instruction-following-post-training-data/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-03-05-instruction-following-post-training-data/</guid><description>LLMs can write poetry and solve math, but ask them to &apos;respond in exactly 3 bullet points using only lowercase&apos; and they stumble. This post dissects the taxonomy of instruction-following failures and provides a practical playbook for building post-training data that actually fixes them.</description><pubDate>Thu, 05 Mar 2026 00:00:00 GMT</pubDate></item><item><title>Experience-Augmented In-Context Learning: A Training-Free Complement to RL Post-Training</title><link>https://ajing.github.io/posts/2026-03-01-experience-augmented-icl-complement-to-rl-post-training/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-03-01-experience-augmented-icl-complement-to-rl-post-training/</guid><description>RL post-training makes models smarter, but it can&apos;t cover the infinite long tail of real-world cases. Experience-augmented ICL retrieves successful reasoning traces at inference time, letting agents learn continuously from real usage — no retraining required.</description><pubDate>Sun, 01 Mar 2026 00:00:00 GMT</pubDate></item><item><title>Tool Selection Optimization for LLM Agents at Scale</title><link>https://ajing.github.io/posts/2026-01-10-tool-selection-optimization-llm-agents-at-scale/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-01-10-tool-selection-optimization-llm-agents-at-scale/</guid><description>A deep technical dive into tool selection—retrieval strategies, context optimization, learned selection, and the engineering trade-offs that matter when scaling to hundreds of tools.</description><pubDate>Sat, 10 Jan 2026 00:00:00 GMT</pubDate></item><item><title>Generative Engine Optimization (GEO): How to Get Your Product Cited by AI</title><link>https://ajing.github.io/posts/2026-01-03-generative-engine-optimization-geo/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-01-03-generative-engine-optimization-geo/</guid><description>A comprehensive guide to Generative Engine Optimization—making your content retrievable, citable, and recommendable by large language models.</description><pubDate>Sat, 03 Jan 2026 00:00:00 GMT</pubDate></item><item><title>Ad Formats in LLM Products: What&apos;s Live vs. What&apos;s Research</title><link>https://ajing.github.io/posts/2026-01-02-ads-formats-in-llm-products/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-01-02-ads-formats-in-llm-products/</guid><description>A survey of advertising formats in LLM products—separating what&apos;s deployed in production from what remains in research.</description><pubDate>Fri, 02 Jan 2026 00:00:00 GMT</pubDate></item><item><title>Adding Ads in LLM/Chatbot: Character Training for Monetization</title><link>https://ajing.github.io/posts/2026-01-02-ads-in-llm-chatbot/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-01-02-ads-in-llm-chatbot/</guid><description>Exploring how to integrate ads in LLMs through character training—making recommendations genuinely helpful rather than annoyingly promotional.</description><pubDate>Fri, 02 Jan 2026 00:00:00 GMT</pubDate></item><item><title>Generative UI Doesn&apos;t Move the Needle—Steering Does</title><link>https://ajing.github.io/posts/2026-01-01-generative-ui-doesnt-move-the-needle-steering-does/</link><guid isPermaLink="true">https://ajing.github.io/posts/2026-01-01-generative-ui-doesnt-move-the-needle-steering-does/</guid><description>After shipping multiple generative UI features, I&apos;ve concluded that the sophistication of AI-generated interfaces often doesn&apos;t translate to user benefit—but steering does.</description><pubDate>Thu, 01 Jan 2026 00:00:00 GMT</pubDate></item><item><title>Post-Training Is Not &apos;One Algorithm&apos;: Objective Functions and Implementation Essentials for PPO / DPO / GRPO</title><link>https://ajing.github.io/posts/2025-12-31-rlhf-ppo-dpo-grpo-notes/</link><guid isPermaLink="true">https://ajing.github.io/posts/2025-12-31-rlhf-ppo-dpo-grpo-notes/</guid><description>Reading notes on RLHF covering PPO, DPO, and GRPO—understanding post-training as an engineering pipeline rather than a single algorithm.</description><pubDate>Wed, 31 Dec 2025 00:00:00 GMT</pubDate></item><item><title>RLHF from an Engineering Perspective: PPO, GRPO, DPO, and Tool-Use Implementation</title><link>https://ajing.github.io/posts/2025-12-31-rlhf-engineering-implementation/</link><guid isPermaLink="true">https://ajing.github.io/posts/2025-12-31-rlhf-engineering-implementation/</guid><description>A practical engineering guide to RLHF implementation—covering PPO, GRPO, DPO, and tool-use training with code snippets and debugging tips.</description><pubDate>Wed, 31 Dec 2025 00:00:00 GMT</pubDate></item><item><title>What Worked (and What Didn&apos;t) When Training AEs and VAEs for Embedding Compression</title><link>https://ajing.github.io/posts/2025-09-07-ae-vae-training-learnings/</link><guid isPermaLink="true">https://ajing.github.io/posts/2025-09-07-ae-vae-training-learnings/</guid><description>Practical lessons from training autoencoders and VAEs for embedding compression—covering dimension choice, KL scheduling, contrastive signals, and evaluation metrics.</description><pubDate>Sun, 07 Sep 2025 00:00:00 GMT</pubDate></item><item><title>User Interest Modeling with Transformer Architectures</title><link>https://ajing.github.io/posts/2025-09-07-user-interest-modeling-with-transformer-architectures/</link><guid isPermaLink="true">https://ajing.github.io/posts/2025-09-07-user-interest-modeling-with-transformer-architectures/</guid><description>Exploring position embeddings, architecture choices, and training techniques for Transformer-based recommender systems.</description><pubDate>Sun, 07 Sep 2025 00:00:00 GMT</pubDate></item><item><title>UI Representation and Action Execution for Generative UI</title><link>https://ajing.github.io/posts/2025-05-03-ui-representation-action-execution/</link><guid isPermaLink="true">https://ajing.github.io/posts/2025-05-03-ui-representation-action-execution/</guid><description>Exploring structured UI representation using JSON Schema, and how to implement action handlers for generative UI systems.</description><pubDate>Sat, 03 May 2025 00:00:00 GMT</pubDate></item><item><title>A Path Towards Generative UI</title><link>https://ajing.github.io/posts/2025-04-01-generative-ui/</link><guid isPermaLink="true">https://ajing.github.io/posts/2025-04-01-generative-ui/</guid><description>Exploring how LLMs can dynamically generate user interfaces that adapt in real time to user needs—the vision behind generative UI.</description><pubDate>Tue, 01 Apr 2025 00:00:00 GMT</pubDate></item><item><title>How to Make LLM Inference Faster</title><link>https://ajing.github.io/posts/2023-08-20-how-to-make-llm-serving-faster/</link><guid isPermaLink="true">https://ajing.github.io/posts/2023-08-20-how-to-make-llm-serving-faster/</guid><description>An overview of LLM inference optimization techniques including KV cache, FlashAttention, and memory management strategies.</description><pubDate>Sun, 20 Aug 2023 00:00:00 GMT</pubDate></item></channel></rss>