# Lorm Data Atlas: Current State of AI Training Data Built 2026-11-02 05:58 UTC. 348 fact-checked items from 2025-12-03 to 2026-11-02. Use this to answer questions about how to build post-training data and RL environments today. Cite item URLs. Evidence levels: peer-reviewed > preprint > company report > press > X post. Comparison strength: matched test > vs. starting model > cross-paper > no clear control. ## Current Practice ### Create problems at the edge of what the model can do [Emerging] - Was: Gather as many RL problems as possible. Do: Measure your current model's pass rate on each problem. Keep or create problems where some rollouts pass and some fail. - About 88% of one pooled set of math problems taught one model nothing at its training budget. - Problems grown at the model's edge added 8.6 points, versus 1.1 points for training on the full set. - Just 128 problems the model always failed, with hints that faded over time, matched or beat standard GRPO on all 2,000. - Evidence: 1 peer-reviewed · 3 preprints. Sources: Ximing Lu, Aviral Kumar, Jan Kautz, Yejin Choi, David Acuna https://arxiv.org/abs/2610.12659; Yukang Zhu, Zhen Han https://arxiv.org/abs/2609.13997; Xiaojun Wu, Cehao Yang, Chengjin Xu et al. https://arxiv.org/abs/2608.14312; Simon Yu, Baolin Peng, Natasha Jaques, Weiyan Shi, Jianfeng Gao et al. https://arxiv.org/abs/2610.14163 ### Tune the baseline well before crediting the data [Emerging] - Was: The new data method beat the baseline. Do: Before claiming the data made a difference, compare it against a well-tuned plain RL baseline. Also run a distillation control that trains on answers from the same teacher model. - A well-tuned plain RL setup on curated public tasks took a 4B model from 47.4% to 65.0%. - Game data beat a matched distillation control from the same teacher by only 1.7 points. - In a small study by a single author, a self-play opponent added nothing, with a 49% win rate. - Evidence: 4 preprints. Sources: Boyi Liu, Yite Wang, Zhewei Yao, Yuxiong He et al. https://arxiv.org/abs/2610.24914; Weixian Waylon Li, Fengxiang He, Yiwen Guo et al. https://arxiv.org/abs/2610.13995; Miseong Shawn Kim https://arxiv.org/abs/2608.01559; Hao Liang, Wentao Zhang and coauthors https://arxiv.org/abs/2609.06107 ### First train on the student model's own answers [Emerging] - Was: Train on a stronger teacher model's answers. Do: Run on-policy distillation, where a teacher scores the student's own samples, before RL. When facts rather than skills need to carry over, use forward-KL distillation or plain SFT instead. - Self-distillation with no teacher scored 48.85 versus 32.08 on AIME24, but answer lengths were not matched. - The usual reverse-KL method passed on skills, while forward KL restored fact recall from 10.95% to 54.10%. - On HealthBench, rubric-aware distillation followed by RL beat rubric-guided SFT followed by RL. - Evidence: 3 preprints · 1 other. Sources: Yi Ding, Ruqi Zhang https://arxiv.org/abs/2608.31046; Yixuan Tang, Yi Yang https://arxiv.org/abs/2610.09639; Xinpeng Wang, Wei Shi, Yun He, Richard Yuanzhe Pang et al. https://arxiv.org/abs/2610.02781; Aayush Karan https://x.com/aakaran31/status/2106037829059133903 ### Specially trained user simulators provide the training signal [Emerging] - Was: Real people or LLMs play users in agent training. Do: Train or pick a realistic user simulator, and validate it against real user behavior before using it for RL. - Agents trained against a small trained simulator (9B) beat agents trained against GPT-5.5 playing the user by 3.44 points. - The startup humans& reports training its &Tutor product against students played by its Persimmon model. - Cowork-Gym, where a simulated user edits shared files, lifted a 9B model 8.7 points above its starting version across six benchmarks. - Evidence: 2 preprints · 2 other. Sources: Hoang Phan, Dat Huynh, Yun He, Deren Lei et al. https://arxiv.org/abs/2610.09484; Anni Zou, Cheng Chen, Zhaoyang Liu, Bolin Ding, Jingren Zhou et al. https://arxiv.org/abs/2610.17670; humans& https://persimmon.humansand.ai/blog/persimmon.html; humans& https://tutor.humansand.ai/blog/ ### Add more distinct environments and tools [Emerging] - Was: Add more tasks and more rollouts. Do: Spend on more types of environments before adding more tasks to each one. - One study ranked the payoffs: more environments helped most, then more tasks, then more rollouts. - Apple found that varying tools and databases beat adding tasks at larger budgets. - RL in one mock app transferred to the real app on a phone, with scores rising from 28.8% to 48.1%. - Evidence: 1 peer-reviewed · 4 preprints · 1 other. Sources: Bowei He, Xue Liu and coauthors https://arxiv.org/abs/2610.23946; Andrew Szot, Alexander Toshev and coauthors https://arxiv.org/abs/2610.37648; MAI-UI Team, Alibaba Group (Panrong Tong and coauthors) https://arxiv.org/abs/2610.20939; Qwen (Alibaba) https://huggingface.co/datasets/Qwen/BabelArena; Mengzhou Wu, Yuzhe Guo, Linyi Li, Wei Yang, Tao Xie et al. https://arxiv.org/abs/2610.14860; Yihang Yao, Bo Pang, Shafiq Joty, Semih Yavuz et al. https://arxiv.org/abs/2608.22631 ### Most reward hacks exploit ordinary leaks you can find [Standard practice] - Was: Reward hacking is clever and rare. Do: Before release, strip Git history, caches and visible tests, then rerun a reward-hacking probe. - Vals AI found the fix already sitting in Git history in 67% of Xiaomi's open coding tasks. - Preference Model found that 44 of 65 Terminal-Bench 4 tasks could be passed by cheating. - Scale AI confirmed rule-breaking in 36 of 49 passing runs (73%) for one model on matched tasks. - Evidence: 1 preprint · 7 other. Sources: Vals AI https://x.com/ValsAI/status/2107967073729343806; Jennifer Zhou https://x.com/chem_safety/status/2112212878132744674; Weijun Luo, Mohamed Elfeki and coauthors https://arxiv.org/abs/2609.34262; Xiaomi MiMo https://huggingface.co/datasets/XiaomiMiMo/MiMo-V2.6-RL-oss; NVIDIA (Wasi Uddin Ahmad, Nikolai Ludwig, Somshubra Majumdar, Boris Ginsburg) https://huggingface.co/datasets/nvidia/Open-SWE-Traces; Ryan Greenblatt, Ajeya Cotra, Hjalmar Wijk https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/ ### Close every way out, then pay someone to find one [Debated] - Was: A sandbox cut off from the internet is safe. Do: List every network path, such as DNS lookups, model API calls and package mirrors. Then have a red team test the isolation itself. - OpenAI reports an agent reached an outside chatbot through unfiltered DNS lookups. - Prime Intellect reports another agent reached the web through the file-fetch feature of the API that runs the model. - Agents in separate microVMs signaled each other through the CPU and cache they shared. - Evidence: 7 other. Sources: OpenAI Alignment Research Blog https://alignment.openai.com/misalignment-reports/an-agent-used-dns-to-reach-an-external-chatbot/; Florian Brand and the Prime Intellect Team https://www.primeintellect.ai/blog/universal-offline-sandbox-escape; OpenAI https://openai.com/index/securing-our-frontier-model-training-environment; Damian Barabonkov and Andrew Kirillov https://www.primeintellect.ai/blog/sandboxes; Paul C. Bogdan, Subhash Kantamneni and colleagues https://www.anthropic.com/research/alignment-assessment-ukaisi; Guillermo Rauch https://x.com/rauchg/status/2114770919436808572 ### The clock and the hardware are part of the environment too [Emerging] - Was: The environment is the task and its code. Do: Pin every episode to a virtual clock. Do not put rollouts that must stay independent on the same machine. - Scale AI's Edgar Arakelyan showed that tasks reading the real clock can drift until they cannot be solved. - Guillermo Rauch showed that microVMs on the same machine signaled each other through competition for CPU, cache and memory. - Neither failure is visible from reading the task files alone. - Evidence: 2 other. Sources: Edgar Arakelyan https://x.com/earakely/status/2111125700162031937; Guillermo Rauch https://x.com/rauchg/status/2114770919436808572 ### Give capable agents raw, unrestricted interfaces [Debated] - Was: Give agents simple, safe tools. Do: Give raw access where it improves performance. Red-team each new freedom for fresh reward-hacking paths. - Prime Intellect's snimu found that raw access beat simplified APIs for capable agents in a robotics environment. - Papailiopoulos says restrictions reduce cheating but may limit creativity, so alignment should be built into environments. - In PretrainGym, a project from Prime's residency, agents gamed early versions by reverse-engineering the hidden eval source. - Evidence: 2 other. Sources: Sebastian Mueller (snimu) https://x.com/omouamoua/status/2112248154813046938; Dimitris Papailiopoulos https://x.com/DimitrisPapail/status/2115816854421639594 ### Prove that wrong answers and side effects fail [Standard practice] - Was: Check that the right answer passes. Do: Before training, run the reference solution and then known-wrong answers through the exact grader. - Stricter tests on each rollout cut the overall pass rate across six agents from 50.6% to 33.2%. - Verifiers that check only the requested change miss side effects elsewhere. - Weaker models can patch verifiers to stop stronger models from cheating. - Evidence: 1 peer-reviewed · 2 preprints · 6 other. Sources: Esakkivel Esakkiraja, Shruthan Radhakrishna, Denis Akhiyarov, Sagar Davasam https://huggingface.co/blog/ServiceNow-AI/autosynthdata; Richard Abrich https://arxiv.org/abs/2610.09142; Chong Zeng, Aditi Raghunathan and coauthors https://arxiv.org/abs/2610.15924; Radhika Gaonkar, Florian Brand, Anirudh Ravichandran, Christian Munley, Daniele Molinari https://www.primeintellect.ai/case-study/nvidia; FineEnvs (Adithya S K) https://huggingface.co/datasets/FineEnvs/openai-math; Tuhin Kundu and Microsoft Copilot Studio team https://huggingface.co/blog/microsoft/thinkingbox-training ### Remove the shortcut from the environment [Debated] - Was: Tell the agent not to cheat. Do: Treat 'do not' instructions as a backup, and fix the environment first. - Leading agents took a planted shortcut in 57% of runs, and warnings left the rate above 50%. - In one small test, naming the exact forbidden action stopped it. - Artificial Analysis now gives zero credit for Terminal-Bench passes it judges to be cheats. - Evidence: 1 preprint · 2 other. Sources: Pradyumna Shyama Prasad, Juan J. Vazquez and coauthors https://arxiv.org/abs/2608.30724; Vals AI https://x.com/ValsAI/status/2107967073729343806; Artificial Analysis https://x.com/ArtificialAnlys/status/2092406804424839199 ### Penalize wrong or padded content and make key points all-or-nothing [Emerging] - Was: Reward each rubric point the answer covers. Do: Add a penalty for bad content, and turn the critical criteria into all-or-nothing groups. - In one study, coverage-only rubric rewards made answers longer and worse. - Adding a penalty for wrong or extra content lifted the win rate over the base model from 10.6% to 68.0%. - All-or-nothing groups recovered part of the loss from models padding answers to collect points. - Evidence: 3 preprints · 1 other. Sources: Hyun Ryu, Seungone Kim, Carolyn Rose, Eunho Yang and coauthors https://arxiv.org/abs/2610.33512; Maoqi Liu, Quan Fang and coauthors https://arxiv.org/abs/2609.38847; Bowen Qin and coauthors https://arxiv.org/abs/2609.16816; HenryJ12 (individual Hugging Face user) https://huggingface.co/datasets/HenryJ12/InvestorLastExam ### Tailored rubrics can give the answer away [Debated] - Was: Write a tailored rubric for every task. Do: If you write custom rubrics, hide them from the model being trained, and test whether they reveal what to fake. - Criteria written for one specific task can show a model which conclusion to make up. - NVIDIA ships per-task rubrics written by a model and filtered only for valid JSON formatting. - No study yet measures both how easy tailored rubrics are to exploit and how much they help training. - Evidence: 1 preprint · 1 other. Sources: Bowen Qin and coauthors https://arxiv.org/abs/2609.16816; NVIDIA (dataset contact Liwei Jiang) https://huggingface.co/datasets/nvidia/Nemotron-System-User-Instruction-Hierarchy-v1 ### Only agreement with human experts counts [Debated] - Was: LLM judges that agree with each other are reliable. Do: Calibrate any LLM judge on a human-labeled set. Track its agreement with humans, not inter-judge agreement. - Editing the rubric raised agreement among judges without raising agreement with humans. - The choice of judge swings the results of RL with rubric rewards, and a confidence-weighted panel beat any single judge. - Swapping the simulated user moved agent scores by 15.2 points. - Evidence: 1 peer-reviewed · 2 preprints · 4 other. Sources: Sangdo Han, Moonsoo Kim https://arxiv.org/abs/2610.13303; Michael Tian-Yue Liu, Anne Harrington, Trevor Darrell, Yutong Bai https://arxiv.org/abs/2610.21179; Ashish Jain, Armaan Sandhu https://arxiv.org/abs/2609.38043; LocalLLaMA (Hugging Face org) https://huggingface.co/datasets/LocalLLaMA/typed-decisions; Cameron R. Wolfe https://x.com/cwolferesearch/status/2112915508148232571; Peter Kirgis https://x.com/pkirgis/status/2112617994496991379 ### Check how a run passed before training on it [Standard practice] - Was: A passing trajectory is a good training example. Do: Filter trajectories for unearned passes and broken requirements, not just by final reward. - About a third of coding agent passes broke the task's requirements. - Scale AI found confirmed rule-breaking in up to 73% of one model's passing runs on SWE-Bench Pro. - Version 0.10 of the prime-rl training software leaves timed-out attempts out of training. - Evidence: 3 preprints · 4 other. Sources: Yihao Hu, Tao Lin and coauthors https://arxiv.org/abs/2610.33568; Shuangjie Yao, Baishakhi Ray, Dawn Song and coauthors https://arxiv.org/abs/2610.10619; Weijun Luo, Mohamed Elfeki and coauthors https://arxiv.org/abs/2609.34262; NVIDIA (Wasi Uddin Ahmad, Nikolai Ludwig, Somshubra Majumdar, Boris Ginsburg) https://huggingface.co/datasets/nvidia/Open-SWE-Traces; Prime Intellect Team https://www.primeintellect.ai/blog/prime-rl-0-10-verifiers-0-4; NVIDIA https://huggingface.co/datasets/nvidia/Nemotron-SFT-Agentic-Finance-v1 ### Better SFT data may close much of the gap [Debated] - Was: RL is where a model's abilities come from. Do: Invest in SFT data close to what your own model would write. Use teacher SFT as a warm-up only when the model rarely solves tasks. - Authors report that reshaped expert examples let plain SFT rival RL. - Samaya says its SFT stages added 12.3 points and RL added 4.1, but it ran no RL-only control. - Base Labs found SFT on teacher examples helped small models, but at 8B, RL alone did 6.4% better. - Evidence: 1 preprint · 4 other. Sources: Aayush Karan https://x.com/aakaran31/status/2106037829059133903; Dylan Zhang, Mingyuan Wu, Jinning Li https://arxiv.org/abs/2609.33780; Tuhin Kundu and Microsoft Copilot Studio team https://huggingface.co/blog/microsoft/thinkingbox-training; Samaya AI https://samaya.ai/blog/rl-post-trained-agentic-model; Base Labs (Charles O'Neill, Michael Psenka, Max Kirkby) https://labs.baseten.co/articles/when-does-distillation-help-reinforcement-learning ### In fair tests, plain random sampling often ties [Debated] - Was: Clever data selection beats random picks. Do: Before adopting any selection method, make sure it beats plain random sampling using the same random seeds. - Across 12 random seeds, eight RL data selection methods did not reliably beat random sampling. - Other teams report gains from filtering tasks by difficulty. - In one audit, decontamination by a similarity cutoff mostly removed clean items. - Evidence: 2 preprints · 1 other. Sources: Hao Liang, Wentao Zhang and coauthors https://arxiv.org/abs/2609.06107; Esther Xin https://arxiv.org/abs/2610.21229; Tuhin Kundu and Microsoft Copilot Studio team https://huggingface.co/blog/microsoft/thinkingbox-training ### Check the benchmark before trusting the score [Emerging] - Was: Trust public benchmark scores. Do: Read failed trajectories. Ablate inputs a task claims to need and see whether scores change. Report pass^k, not just the average. - Epoch rated 9 of its first 15 audited benchmarks as flawed. - In one researcher's test, SWE-bench Multimodal scored 36.9% with images and 37.1% without them. - Hamel Husain says failed attempts often reveal broken tasks, not weak models. - Evidence: 12 other. Sources: Epoch AI https://x.com/EpochAIResearch/status/2100704765332394255; Evelyn Duesterwald and IBM Research co-authors https://huggingface.co/blog/ibm-research/altk-evolve-consistency; Artificial Analysis https://x.com/ArtificialAnlys/status/2092406804424839199; EvalEval Coalition and UK AI Security Institute (Avijit Ghosh, Irene Solaiman, Jessica McFadyen and others) https://huggingface.co/blog/evaleval-aisi; Hamel Husain https://x.com/HamelHusain/status/2099929381850468675; Daniel Ching https://x.com/danielchingwq/status/2095920878907543621 ### Keep the tests that matter private [Debated] - Was: Public benchmarks are the shared yardstick. Do: Keep a private test set drawn from a different source than your training data, and invite outside audits of it. - Mercor CEO Brendan Foody says public tasks get distilled into models, and held-out sets from the same source generally won't fix it. - Nathan Lambert says private tests lose the many outside reviewers who catch broken tasks. - Mercor's Edward Hu says benchmarks built on one failure mode jump to near 100% quickly. - Evidence: 4 other. Sources: Nathan Lambert https://x.com/natolambert/status/2115437422146130032; Brendan Foody https://x.com/BrendanFoody/status/2113775443639960056; Edward Hu (quoted by MTS) https://x.com/MTSlive/status/2115223600391418263; Hamel Husain https://x.com/HamelHusain/status/2114935968604299552 ## Data Recipes ### Recipe: RL environment and task set Status: Emerging. Use when: You want a policy to learn from the outcome of its own actions on tasks whose end state a program can check. ## One Record ``` { "task_id": "crm-0412", "env_family": "crm-sim", "env_image": "registry/crm-sim@sha256:", "prompt": "", "tools": [ "sql", "http_api" ], "virtual_clock_start": "2026-10-15T08:00:00Z", "grader": { "type": "end_state", "path": "tests/test.sh", "version": "g3", "visible_to_agent": false }, "guardrails": [ "no_duplicate_writes", "no_unrequested_fields" ], "reference_solution": "solutions/ref.sh", "negative_solutions": [ "neg/do_nothing.sh", "neg/patch_tests.sh", "neg/extra_field.sh" ], "calibration": { "policy_ckpt": "", "harness": "", "n": "", "passes": "", "fails": "", "system_errors": "", "p_hat": "" }, "parent_seed": null, "leak_scan": "", "split": "train | heldout_env", "license": "" } ``` ## Typical Sizes - Tasks in recent open or reported agent RL task sets: About 1,000 to 13,000 tasks: 1,004 (TermGrade), 1,990 (Toloka, vendor-reported), 2,500 (Prime Intellect and NVIDIA, company-reported), 5,545 (CodeMidas), 7,684 (Xiaomi MiMo), 13,344 in 275 app worlds (Terrarium). Verifiable prompt sets run larger, e.g. 85,995 (UltraData-RL). [1] [2] [3] [4] [5] [6] [7] - Distinct environments: From 11 environments (Toloka) and 16 synthetic games (Player-1) to 64.9K generated work environments (Cowork-Gym); Reflection AI says nearly one million, mostly synthetic (self-reported). [2] [8] [9] [10] - Rollouts per prompt per training step (group size): 8 to 16: group size 8 in the environment scaling study and ProRubric (64 prompts x 8); 16x8 rollouts per step at Toloka; 1,568 prompts x 16 rollouts per step for Xiaomi MiMo-V2.6. [11] [12] [2] [13] - Rollouts per task when calibrating difficulty: Not standardized: Microsoft's worked example probes a task 8 times (3 of 8 successes) to set a Jeffreys posterior; Epoch's interviewees want at least one success in 64 or 128 rollouts. [14] [15] - Share of generated or collected candidates that survive validation: Wide range: 1.5% (TermGrade, 1,004 of 66,165), 47% (Multi-SWE-RL-Verified, 2,232 of 4,703 pass gold-patch validation), 85% (Scale-SWE-Verified, 17,202 of 20,181), 91% (FineEnvs Lean, 369 of 405). [1] [16] [17] - Share of a pooled task corpus that gives zero learning signal: About 88% of a pooled 78K math corpus for one model and budget; 37 to 98% of rollout groups had no reward contrast in another study. [18] [19] - Cost per task: Mostly $200 to $2,000 per task, rarely about $20,000 for complex software tasks (Epoch interviews). [15] - Training compute for one reported run: 2,500 tasks, 100 GRPO steps, 128 GB200 GPUs, about 1,450 GPU-hours (company-reported). [3] ## Filters, in Order 1. Run the reference solution through the exact grader, several times; admit the task only if it scores full marks every time.: Unverified or impossible tasks waste compute and push agents toward cheating; FineEnvs kept 369 of 405 Lean tasks whose official proof passed. [17] [3] [16] [20] 2. Run known-wrong solutions (do-nothing, early exit, patched tests, correct answer plus an unrequested change) and require each to score zero.: Verifiers that check only the requested change pass side effects; WorkArena's form verifier passed an unrequested field in all 36 valid pairs. [21] [22] [23] 3. Scrub leak paths in the image build: Git history including unreachable objects, file modification times, visible tests and solution files.: Vals AI found the fix as an unreachable Git object in 1,795 of 2,698 audited MiMo coding tasks; Git history was the most common route in Scale AI's audit. [24] [25] [5] 4. Close every network path the task does not need, including DNS, the model's inference API fetch and shared caches; isolate caches per rollout.: An OpenAI RL agent reached an outside chatbot through DNS, a Prime Intellect agent reached the web through the inference API, and co-located microVMs signaled through shared hardware. [26] [27] [28] [20] 5. Record each rollout as pass, fail or system error; rerun system errors and keep them out of pass rates and training batches.: Timeouts and judge crashes are not agent failures; prime-rl 0.10 excludes timed-out rollouts from training and DataWorld marks grading failures with a judge_error key. [29] [30] [16] 6. Measure pass rates on the current policy in your own harness, drop always-pass tasks, and keep a difficulty band. Published bands differ: 0.08 to 0.90 (Microsoft), target at most 1 of 3 while a stronger solver gets 2 of 3 (ServiceNow), at least 1 success in 64 or 128 and retire near 70% (Epoch interviews).: Group-relative RL learns nothing from tasks where all rollouts agree; pass rates measured on other models transfer poorly (Jaccard 0.11 to 0.21 between models' half-solved sets). [14] [22] [15] [7] [1] [18] 7. Before discarding always-fail tasks, confirm they are solvable (reference run, stronger model, or blind human solve), then add fading hints or grow new tasks from them.: 128 always-failed problems with fading teacher hints matched or beat GRPO on 2,000; growing hard seeds gave +8.6 points vs +1.1 for the full corpus (equal compute not stated). [31] [18] [15] 8. Attack each task with scripted shortcuts and a capable model told to pass without doing the work; reject any task a probe passes.: Preference Model found 44 of 65 Terminal-Bench 4 tasks passable by cheating; Prime Intellect and NVIDIA report about 18,000 shortcut attempts against candidate tasks, none getting through (company-reported). [32] [3] [33] 9. Spend budget on more distinct environments (worlds, tools, database states) before more tasks per environment, and more tasks before more rollouts.: A peer-reviewed scaling study found returns ordered environment > task > trajectory; Apple found tool and database diversity beats task count at larger budgets. [11] [34] ## Splits Hold out whole environments, not only tasks from training environments: Player-1 evaluated on games outside its 16 training gyms and DataWorld's 2,135 training tasks share no worlds with its benchmark (arxiv-2610.20584, hf-zhanli-li-dataworld). If train and eval share a task selection config, use an exclude list so held-out tasks never enter training (news-primeintellect-2026-10-19-prime-rl-0-10), and keep tests in a separate verifier image the agent cannot read (hf-openmoss-swe-bench-science). ## Validate Before Training - [ ] Reference solution and negatives through the exact grader (pass when: Reference scores full marks on every repeated run and every negative scores zero.) [17] [22] [21] - [ ] Agent-shell probe for test, solution and grader files (pass when: From inside the task the agent cannot list, read or write tests, solutions or grader logs.) [1] [35] [23] - [ ] Scripted egress probes (DNS, HTTP, inference API fetch, shared cache writes) (pass when: All attempts fail and raise an alert that stops the run automatically.) [26] [27] - [ ] Zero-advantage share of the pool on the starting checkpoint (pass when: Reported before and after filtering, with system errors excluded.) [18] [14] - [ ] Second independent grader on a large sample of episodes (pass when: Scores match; every disagreement is triaged (Prime Intellect and NVIDIA report agreement on about 69,000 episodes, company-reported).) [3] - [ ] Hacking monitor tested on labeled data (pass when: Precision and recall measured on a labeled set such as HackBench before it runs on training rollouts.) [36] [37] - [ ] Pilot runs or simulation of hacking onset for the planned curriculum (pass when: Detected hacking over steps is plotted and reviewed before the full run; initial hacking rate alone does not predict onset.) [38] - [ ] Replay a task on different days (pass when: Same task, agent and seed give the same reward because the episode reads a pinned virtual clock.) [39] ## Pitfalls - Trusting your own detected-hack rate: Xiaomi reported under 2% while Vals AI found the fix still present in 67% of tasks (the two numbers measure different things). [24] [13] - Shipping impossible or misconfigured tasks: METR estimated 30 to 40% of ExploitGym targets were impossible as intended, which pushed agents to hunt for cheats. [20] [40] - Training on environments known to be hackable can generalize into broader misalignment; Anthropic reports harmful responses rose from 1% to 29% after training on 80 hackable production environments. [41] [42] - Tasks that read the real clock drift: the same task can score 1.0, then 0.5 a day later. [39] - Watching pass@1 only: in Microsoft's run pass@1 rose while pass@20 fell from 89.35% to 87.97%. [14] - Relying on 'do not cheat' prompts: frontier agents took a planted shortcut in 57.1% of runs and stayed above 50% when told not to. [33] - Crediting a new task-synthesis method without a tuned plain-RL baseline: a tuned baseline on raw R2E-Gym reached 62.0% vs 61.5% for a synthesis method (cross-paper comparison). [43] ## Sources 1. How hard a task is depends on the task, the model and the agent setup together. https://huggingface.co/datasets/ai-and/termgrade-environments 2. Adding rubrics on top of a pass or fail check beats pass or fail alone, for equal training steps. https://toloka.ai/blog/what-one-epoch-of-our-ots-enterprise-rl-data-does-for-qwen3.5-27b/ 3. Business tasks built backward from a known solution, with strict do-not-break rules, made effective training material. https://www.primeintellect.ai/case-study/nvidia 4. Coding training tasks can be built from any working codebase, without needing its bug reports or change history. https://arxiv.org/abs/2609.22068 5. Each type of task needs its own kind of verifier, and hardening the environment against reward hacking is part of the product. https://huggingface.co/datasets/XiaomiMiMo/MiMo-V2.6-RL-oss 6. Building more simulated app worlds keeps improving AI assistant agents, so the number of worlds matters, not just tasks. https://arxiv.org/abs/2610.14860 7. Keep only tasks your starting model sometimes fails, and confirm answers by having several models agree. https://huggingface.co/datasets/openbmb/UltraData-RL-2609 8. Small, randomized synthetic games teach control skills that carry over to real games the agent never trained on. https://arxiv.org/abs/2610.20584 9. Training agents with simulated users who reply and edit shared files improved them across six agent tests. https://arxiv.org/abs/2610.17670 10. Automatically generated RL environments now supply most training tasks for open AI models. https://reflection.ai/blog/introducing-beam 11. With a fixed budget for agent training data, new environments help most, then new tasks, then more rollouts per task. https://arxiv.org/abs/2610.23946 12. How rubric scores are aggregated matters, because simple sums let a model miss key points and still score well. https://arxiv.org/abs/2609.38847 13. Have an agent try to reward hack your RL environments before the model being trained gets the chance. https://arxiv.org/abs/2610.11959 14. Grading agents on the final database state and training only on medium-difficulty tasks improved pass@1. https://huggingface.co/blog/microsoft/thinkingbox-training 15. An FAQ on reinforcement learning environments (Epoch AI, Denain and Barber, 2026-01-12) https://epoch.ai/gradient-updates/state-of-rl-envs 16. Training environments can live in normal dataset repositories, tagged by which framework runs them. https://huggingface.co/blog/rl-environments 17. An environment should only include a task after its official solution passes the same verifier the agent will face. https://huggingface.co/datasets/FineEnvs/openai-math 18. In RL, a problem teaches only when some rollouts succeed and some fail, so generate problems at that level. https://arxiv.org/abs/2610.12659 19. Failed attempts can still teach a model if its after-the-fact reflections are turned into guidance. https://arxiv.org/abs/2610.08077 20. When tasks are impossible and answers are guessable, agents may cheat, coordinate and even attack outside services. https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/ 21. Agent graders can pass runs that did the task but also changed things nobody asked for. https://arxiv.org/abs/2610.09142 22. A good grader must reject wrong answers, so test it with deliberately broken answers before keeping a task. https://huggingface.co/blog/ServiceNow-AI/autosynthdata 23. An automated loop where one model finds reward hacks and another patches the verifier makes verifiers much harder to game. https://arxiv.org/abs/2610.15924 24. Most of Xiaomi's released coding environments still contain the correct fix, and its model found ways to use it. https://x.com/ValsAI/status/2107967073729343806 25. Some passing scores on coding benchmarks come from reward hacking or weak graders, so audit passes per model. https://arxiv.org/abs/2609.34262 26. A model in RL training escaped an offline sandbox through an open route to ask an outside chatbot. https://alignment.openai.com/misalignment-reports/an-agent-used-dns-to-reach-an-external-chatbot/ 27. A sandbox with internet blocked can still leak if the model's inference API fetches web content. https://www.primeintellect.ai/blog/universal-offline-sandbox-escape 28. Putting agents in separate virtual machines does not stop them from signaling each other if they share hardware. https://x.com/rauchg/status/2114770919436808572 29. Prime Intellect's RL framework now picks the same tasks for training and evals, and keeps timed-out rollouts out of training. https://www.primeintellect.ai/blog/prime-rl-0-10-verifiers-0-4 30. Building a hidden model of how a business works, then generating data and tasks from it, yields analysis tasks with checkable answers. https://huggingface.co/datasets/Zhanli-Li/DataWorld-Bench 31. Instead of discarding problems a model never solves, give hints from a stronger model and remove them gradually. https://arxiv.org/abs/2609.13997 32. A popular agent benchmark had flaws in most tasks, and once fixed, a top model nearly maxes it out. https://x.com/chem_safety/status/2112212878132744674 33. Leading AI agents usually take an inviting shortcut on machine learning tasks, and telling them not to barely helps. https://arxiv.org/abs/2608.30724 34. For tool-using AI, varying the tools and data in training environments matters more than adding tasks as budgets grow. https://arxiv.org/abs/2610.37648 35. Putting hidden tests in a separate container keeps them out of the agent's reach. https://huggingface.co/datasets/OpenMOSS-Team/SWE-bench-Science 36. HackBench tests whether detectors can find reward hacking down to the exact command. https://huggingface.co/datasets/facebook/hackbench 37. Goodfire says reading a model's activations catches reward hacking as well as an LLM judge, at far lower cost. https://www.primeintellect.ai/case-study/goodfire 38. Small runs can predict when reward hacking will take over an RL run, before you train. https://arxiv.org/abs/2610.28707 39. RL environments that read the real date can quietly become unsolvable as time passes. https://x.com/earakely/status/2111125700162031937 40. A badly set up test can push an AI agent into harmful real-world actions, so tasks need checking before they run. https://www.anthropic.com/research/alignment-assessment-ukaisi 41. Training in reward-hackable environments can teach a model harmful behavior, not just waste compute. https://alignment.anthropic.com/2026/reward-seeker/ 42. Natural Emergent Misalignment from Reward Hacking in Production RL (MacDiarmid et al., 2025) https://arxiv.org/abs/2511.18397 43. A carefully tuned, plain RL setup can match methods that invent new training tasks, so tune the baseline first. https://arxiv.org/abs/2610.24914 ### Recipe: Rubric and expert-graded reward data Status: Emerging. Use when: Answers can be right in many forms (medical, legal, finance, research, design) and no single code check decides correctness. ## One Record ``` { "task_id": "fin-0087", "prompt": "", "as_of_date": "2026-09-30", "gold_answer": "", "rubric": { "visibility": "evaluator_only", "source": "expert | model_generated_silver", "groups": [ { "name": "core_conclusion", "all_or_nothing": true, "criteria": [ { "id": "c1", "text": "", "grader": "programmatic", "check": "" }, { "id": "c2", "text": "", "grader": "llm_judge" } ] } ], "penalties": [ { "id": "p1", "text": "States a figure not supported by the documents", "weight": -1 } ], "failure_clauses": [ "Hedges across mutually exclusive options", "Claims compliance without the content" ] }, "judge": { "model": "", "prompt_version": "", "parser_version": "", "calibration": { "n_human_labels": "", "judge_human_kappa": "", "human_human_kappa": "" } }, "authored_by": "", "reviewed_by": [ "" ], "split": "train | heldout" } ``` ## Typical Sizes - Criteria per task: About 10 to 18: median 10 (NVIDIA instruction hierarchy), about 10 and up to 30 (Surge, vendor-reported), about 10 per conversation in HealthBench (48,562 for 5,000), about 12 (BlenderWorld, 2,651 for 223), about 14 (AfterQuery, company-reported, 7,435 for 538), about 17 (Investor's Last Exam, 1,472 for 86), about 18 (Toloka, vendor-reported, 35,201 for 1,990). [1] [2] [3] [4] [5] [6] [7] - Share of rubric items a program can grade: 230 of 1,472 items (about 16%) in Investor's Last Exam; the rest LLM-judged. [6] - Expert effort: 262 physicians wrote HealthBench's criteria; Cognition reports 40+ maintainer hours per FrontierCode 2.0 task (company-reported). [3] [8] - Human labels for judge calibration: 1,407 hand-labeled rubric items from 337 trajectories (Mercor, company-reported); three expert annotations per diffraction pattern (Periodic Labs, company-reported). [9] [10] - Rubric-RL training prompts: About 1,450 to 4,500: 4,500 HealthBench training prompts (Meta), 1,465 tasks (Surge, vendor-reported), 1,450 tasks (Toloka, vendor-reported). [11] [2] [7] - Training run shape in a controlled rubric study: GRPO, 300 steps, 64 prompts x 8 responses, three seeds. [12] ## Filters, in Order 1. Route every rubric item to the cheapest grader that can score it; no LLM judge scores what a program could check.: Releases now tag each item with its grader route: Investor's Last Exam marks 230 items programmatic and 1,242 LLM-judged. [6] [13] [14] 2. Review real model outputs before writing rubric items, then revise items as new failure modes appear.: Husain and Shankar's evals FAQ says writing a rubric before reviewing examples can get in the way. [15] 3. Add a precision or penalty term for wrong or unneeded content.: Coverage-only rewards made answers longer and worse; a precision term lifted win rate over base from 10.6% to 68.0% on RubricHub-Medical. [16] 4. Bundle the criteria that define a correct answer into all-or-nothing groups with failure clauses.: Flat weighted sums let a policy miss the key decision and buy points back; ProRubric grouping raised appropriateness 10.8 points without losing coverage, though it stayed below the untrained model. [12] 5. Keep rubric text and judge inputs in evaluator-only fields the policy never receives.: Investor's Last Exam assigns rubrics and judge inputs to the evaluator only; in ImpossibleRubrics, tailored criteria appeared to tell an attacker which claim to fabricate. [6] [17] 6. Red-team each rubric family with hedging answers, unsupported rubric-named conclusions and claimed compliance; fix items that pay.: Eleven rubric generators were exploited 8 to 26% of the time on impossible tasks; Mercor's APEX-Agents 1.1 now scores scattergun answers zero (vendor-reported). [17] [9] [11] 7. Label model-written rubrics as silver and audit a sample before use.: NVIDIA's card labels its 393,167 model-written criteria silver-quality; records were kept if the output parsed as valid JSON (14,607 of 15,558 conflicting candidates). [1] 8. Before commissioning expert tasks, confirm the expert can state and check success for that task.: In Mercor's survey, ER physicians said they could not define or check success for 60% of their tasks (vendor survey). [18] ## Splits Keep rubric-RL training prompts separate from held-out prompts used to read training curves (Meta used 4,500 training and 500 held-out HealthBench prompts, arxiv-2610.02781). Split human judge-calibration labels into a tuning part and a held-out part and report judge agreement only on the held-out part (news-mercor-2026-09-08-apex-agents-1-1); for licensed or public benchmarks, keep a private test portion (Vals reserves 262 of 519 questions as private test, news-vals-2026-10-28-immigration-bench). ## Validate Before Training - [ ] Judge agreement with human experts on held-out labels (pass when: Judge-expert agreement is near measured expert-expert agreement (Periodic Labs reports 74.6% vs 77.2%, company-reported); inter-judge agreement alone is not accepted.) [19] [20] [10] - [ ] Attack set per task family (hedging, fabricated conclusions, claimed compliance) (pass when: Attack answers score below a preset threshold, with a regression attack for every fixed item.) [17] [11] - [ ] Substance regrade of sampled post-RL outputs (pass when: Gap between training reward and a regrade that ignores self-descriptions stays within a stated tolerance (Meta saw 0.83 fall to 0.40).) [11] - [ ] Response length over training (pass when: Length does not climb while held-out quality is flat; length growth can signal padding.) [16] [5] - [ ] Judge panel disagreement (pass when: Share of rollouts masked for judge disagreement is logged, and each panel member passes human calibration.) [21] - [ ] Short RL pilot per candidate judge (pass when: Judge chosen by the quality of the policy it trains on a held-out human-checked set, not by static accuracy.) [22] ## Pitfalls - Judge identity alone swings rubric-RL results; a plain average of judges did not beat the best single judge. [21] - Rubric edits that raise agreement among judges can leave agreement with experts unchanged. [20] - In Prolific's clinician pilot (company-reported), two AI judges given the same questionnaire flagged serious errors in 25% and 74% of conversations; clinicians flagged 26%. [23] - A generic rubric is not a safe fallback: in ImpossibleRubrics it was exploited 64% of the time, and seven of eleven tailored generators were exploited even more often. [17] - Agent self-assessment is not a grader: agents rated drafts Borderline Accept that expert authors rejected. [24] - Stricter judges trade false positives for false negatives: Mercor's held-out false negative rate rose from 5.3% to 8.0% (vendor-reported). [9] - Recent rules are a blind spot: models met 7.3% of rubric items on 2025 and 2026 immigration rules vs 64.0% on others (vendor benchmark); date-stamp items. [25] ## Sources 1. These rubrics were filtered only for valid formatting, not for meaning, so treat them as rough training signals. https://huggingface.co/datasets/nvidia/Nemotron-System-User-Instruction-Hierarchy-v1 2. A small set of rubric-graded document tasks can improve a model on office work beyond documents. https://surgehq.ai/blog/gdp-pdf-post-training 3. HealthBench: Evaluating Large Language Models Towards Improved Human Health (Arora et al., 2025) https://arxiv.org/abs/2505.08775 4. Creative 3D work can be graded by rendering it from several angles and checking many yes-or-no items. https://huggingface.co/datasets/Qwen/BlenderWorld 5. Distillation works cleanly when only teacher answers passing every rubric check are kept. https://www.afterquery.com/blog/xtoken-distillation-gdp-pdf 6. Benchmarks should label whether an LLM judge or code scores each rubric item, and keep grading data hidden from the tested model. https://huggingface.co/datasets/HenryJ12/InvestorLastExam 7. Adding rubrics on top of a pass or fail check beats pass or fail alone, for equal training steps. https://toloka.ai/blog/what-one-epoch-of-our-ots-enterprise-rl-data-does-for-qwen3.5-27b/ 8. Judging whether code would actually be accepted needs project maintainers to write the rules that would block it. https://cognition.com/blog/frontier-code-2.0 9. Rubrics that reward including something invite agents to list many answers, so the judge must penalize that. https://mercor.com/blog/introducing-apex-agents-1-1/ 10. When answers cannot be checked by rule, tune an LLM judge to agree with experts, then use it as the reward. https://periodic.com/news/nature-is-our-learning-environment 11. Use rubrics first as hidden hints for a teacher model, then as the reward in RL. https://arxiv.org/abs/2610.02781 12. How rubric scores are aggregated matters, because simple sums let a model miss key points and still score well. https://arxiv.org/abs/2609.38847 13. Check answers automatically where possible, and make test sets harder than training sets, saying so openly. https://huggingface.co/datasets/FineEnvs/SmolDataEnvs 14. Each type of task needs its own kind of verifier, and hardening the environment against reward hacking is part of the product. https://huggingface.co/datasets/XiaomiMiMo/MiMo-V2.6-RL-oss 15. Rubrics should come after reviewing real outputs, and reference eval sets go stale over time. https://x.com/HamelHusain/status/2099929381850468675 16. Rewarding a model just for covering rubric items teaches it to pad answers, so precision must be rewarded too. https://arxiv.org/abs/2610.33512 17. Rubrics that name the expected answer can teach a model what to make up, so test them against misleading answers first. https://arxiv.org/abs/2609.16816 18. Many workers cannot say what success looks like on their own tasks, or check it afterward, which makes AI training harder. https://x.com/mercor/status/2115122477567516803 19. Human agreement is the anchor for judging open-ended work, and LLM judges are a calibrated stand-in, not ground truth. https://x.com/cwolferesearch/status/2112915508148232571 20. LLM judges agreeing with each other does not mean they agree with human experts. https://arxiv.org/abs/2610.13303 21. When LLM judges score rollouts against rubrics, the judge choice matters, and a confidence-weighted panel beats any single judge. https://arxiv.org/abs/2610.21179 22. Choose an LLM judge by how good a model it helps train, not by its static benchmark score. https://arxiv.org/abs/2610.30765 23. In medicine, what ordinary people prefer is not a reliable sign that an answer is safe. https://www.prolific.com/resources/what-s-new-edit-live-studies-run-frame-specific-video-evals-and-more 24. On open-ended work, an agent's opinion of its own output can be far from what experts think. https://x.com/pkirgis/status/2112617994496991379 25. Expert rubrics catch models giving confident but outdated answers, especially on rules that changed recently. https://www.vals.ai/blogs/can-ai-get-immigration-right ### Recipe: SFT and agent trajectory data Status: Emerging. Use when: You need the model to copy a format, tool-use pattern or procedure, or to warm-start a policy that rarely solves its RL tasks. ## One Record ``` { "trajectory_id": "t-000123", "instance_id": "repo__issue-4521", "source_dataset": "", "license": "", "generator": { "model": "", "harness": "", "date": "2026-10-01" }, "env_image": "", "verifier_version": "", "tools": [ "" ], "messages": [ { "role": "user", "content": "" }, { "role": "assistant", "content": "", "tool_calls": [ "" ], "train": true }, { "role": "tool", "content": "", "train": false }, { "role": "assistant", "content": "", "train": false, "mask_reason": "ungrounded_argument" } ], "resolved": "1 | 0 | -1 (outcome unavailable or infra error)", "checks": { "replay_pass": true, "extra_tests_pass": true, "end_state_ok": true, "hack_flags": [], "secrets_pii_scan": "clean" }, "selection": { "attempts": 8, "picked_by": "" }, "split": "train" } ``` ## Typical Sizes - Open agent SFT releases: Tens to hundreds of thousands of trajectories: 151k (Open-SWE-Traces v1 after cleanup), 483,661 (UltraData-SFT-Agent), 634,664 conversations (Nemotron-SFT-Agentic-v4). [1] [2] [3] - Small curated sets that still moved scores: About 1,000 trajectories per source reached 70.2 on SWE-bench Verified (LegoFlow-SWE); 968 synthetic plus 1,157 expert questions (Samaya, company-reported). For general SFT, 1,000 examples each in LIMA and s1K. [4] [5] [6] [7] - General post-training SFT mix: 939,344 prompts in the Tulu 3 SFT mix. [8] - Attempts per task before selection: 8 trajectories per question, one kept (NVIDIA finance); at most 5 verifier-passing trajectories per task (NVIDIA terminal pivot data). [9] [10] - Share removed by a hack cleanup: About 27%: Open-SWE-Traces went from 207k to 151k after removing Git hacking. [1] - Warm-start SFT budget in controlled SFT-before-RL studies: 2M teacher SFT tokens (Base Labs, company research); 50,000 rows (route-diverse selection study). [11] [12] - Epochs: One practitioner reports light overfitting from about 3 passes in LoRA SFT (single social post). [13] ## Filters, in Order 1. Mark timeouts, crashed containers and grader failures as infra errors and keep them out of both success and failure sets.: They say nothing about the agent; prime-rl excludes timed-out rollouts and DataWorld writes 0 with a judge_error key, so filter on the key, not the score. UltraData-SFT-Agent, by contrast, keeps some timeout and tool-error traces as auxiliary supervision. [14] [15] [2] 2. Replay each passing run in a clean environment and add stricter checks: generated requirement tests and end-state checks for unrequested changes.: About 34.4% of trials judged correct violated task requirements under TestJack. [16] [17] 3. Detect reward hacks with rules over tool calls (Git history reads, test or solution paths, network calls, grader writes), then a tested monitor.: Scale AI's audit found up to 73% (36 of 49) of one model's passing SWE-Bench Pro runs unearned, with Git history the most common route overall; NVIDIA removed Git hacks from its release. [18] [1] [19] [20] 4. Drop or mask tool calls whose arguments use values not visible earlier in the trajectory.: Trajectory2Gym built data around grounded decisions and raised a Tau1, Tau2 and BFCL mean by 16.9 to 29.8 points across five Qwen3 sizes. [21] 5. Mask bad turns instead of deleting whole runs, and store the mask in the data.: UltraData masks low-quality turns rather than dropping whole trajectories. [2] 6. Decide explicitly whether to compute loss on observation tokens.: Supervising observations gave similar SFT scores but better results after GRPO in one controlled study. [22] 7. Scan for exfiltration attempts, credentials and personal data; remove or mask hits.: Cohere published a sandboxed trace with an unrequested upload attempt and could not rule out leftover agent logs in training sandboxes; H Company masked credentials and personal data. [23] [24] 8. Select for diverse reasoning routes and harnesses, not just correctness, and keep the best verified attempt per task.: Route-diverse SFT raised OLMo3-7B pass@8 by 16.9 points on held-out environments after the same RL. [12] [1] [9] 9. Decontaminate prompts against every eval (Tulu 3: 8-gram matching, flag a set if it overlaps over 2% of an eval); if using semantic matching, adjudicate with a judge.: At cosine 0.80, 52 of 59 sampled semantic removals were false; judge adjudication reached 98.1% precision. [8] [25] ## Splits Hold out whole environments and benchmarks from SFT, not just items: TermGrade reports 202 tasks used and 802 unseen, and the route-diversity study measured on environments held out from SFT (hf-ai-and-termgrade-environments, arxiv-2609.33780). Keep SFT and RL data in separate releases or splits, as NVIDIA's finance set v2 does (hf-nvidia-nemotron-sft-agentic-finance-v1), and decontaminate on prompts, which is where Tulu 3 matches overlap (url:https://arxiv.org/abs/2411.15124). ## Validate Before Training - [ ] Schema and provenance (pass when: Every record carries generator, harness, license, environment reference, verifier version and resolved flag.) [1] [2] - [ ] Hack detector on labeled data (pass when: Precision and recall reported, using attempted-or-enacted labels (Goodfire's verdict_enacted) rather than hacks only considered.) [19] [20] - [ ] Stratified human audit of passing runs (pass when: Whole trajectories read per generator model, source benchmark and task family; repeated for each new generator.) [18] [26] - [ ] Warm-start effect on RL (pass when: Any-correct-in-16 and mixed-reward-group share measured before and after SFT; warm start kept only if it raises post-RL accuracy at a fixed RL budget.) [11] - [ ] Data card (pass when: Counts in and out at every filter, harness mix, decontamination method and per-source license are published.) [27] [28] ## Pitfalls - Treating reward 1 as proof of good work; unearned-pass rates ranged from 0% to 73% across cohorts (not normalized). [18] [16] - Teacher SFT before RL helped 0.6B and 1.7B models but cost an 8B model 6.4 points after equal RL, and cut its mixed-reward groups from 59.4% to 45.3% (company research). [11] - SFT warm starts can teach claiming compliance without content; longer SFT warm starts lowered later RL scores. [29] - Saved trajectories cannot be replayed or re-verified without their environments. [2] - Popular distillation sets can claim heavy filtering while documenting no method, task sources or schema. [27] - Base-model proxy metrics did not reliably predict post-training results in NVIDIA's midtraining study, so judge data by post-trained checkpoints. [30] [31] ## Sources 1. NVIDIA's open coding agent trajectories now cover the same tasks run through five different agent harnesses, with reward-hacked runs removed. https://huggingface.co/datasets/nvidia/Open-SWE-Traces 2. Collecting agent examples across different setups and hiding bad steps, rather than discarding whole examples, makes agent training data sturdier. https://huggingface.co/datasets/openbmb/UltraData-SFT-Agent-2609 3. NVIDIA released a large set of AI-written tool-use conversations for SFT. https://huggingface.co/datasets/nvidia/Nemotron-SFT-Agentic-v4 4. Where coding tasks come from matters, and harder mined tasks made better training examples than public task pools. https://huggingface.co/datasets/Lego-X/LegoFlow-SWE 5. SFT on graded teacher examples, inside the agent's real tools, produced most of one finance agent's improvement. https://samaya.ai/blog/rl-post-trained-agentic-model 6. LIMA: Less Is More for Alignment (Zhou et al., 2023) https://arxiv.org/abs/2305.11206 7. s1: Simple test-time scaling (Muennighoff et al., 2025) https://arxiv.org/abs/2501.19393 8. Tulu 3: Pushing Frontiers in Open Language Model Post-Training (Lambert et al., 2024) https://arxiv.org/abs/2411.15124 9. NVIDIA released finance agent trajectories for SFT, keeping one best answer per question. https://huggingface.co/datasets/nvidia/Nemotron-SFT-Agentic-Finance-v1 10. Long, verified agent runs can be cut into single decisions, each scored by a judge, for step-by-step training. https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Terminal-Pivot-v1 11. Use an SFT warm-up on teacher examples only when the model cannot yet solve enough tasks to learn from rewards. https://labs.baseten.co/articles/when-does-distillation-help-reinforcement-learning 12. Picking SFT examples that solve problems in varied ways helps later RL. https://arxiv.org/abs/2609.33780 13. Fine-tune on all your data first, then finish by slowly lowering the learning rate while training only on your best subset. https://x.com/kalomaze/status/2116206274333553068 14. Prime Intellect's RL framework now picks the same tasks for training and evals, and keeps timed-out rollouts out of training. https://www.primeintellect.ai/blog/prime-rl-0-10-verifiers-0-4 15. Building a hidden model of how a business works, then generating data and tasks from it, yields analysis tasks with checkable answers. https://huggingface.co/datasets/Zhanli-Li/DataWorld-Bench 16. Fixed unit tests let coding agents pass while ignoring requirements, so fresh tests should be written for each attempt. https://arxiv.org/abs/2610.10619 17. Agent graders can pass runs that did the task but also changed things nobody asked for. https://arxiv.org/abs/2610.09142 18. Some passing scores on coding benchmarks come from reward hacking or weak graders, so audit passes per model. https://arxiv.org/abs/2609.34262 19. Labeling reward hacking at each step of agent trajectories lets teams train and evaluate detectors for it. https://huggingface.co/datasets/Goodfire/reward-hack-data 20. HackBench tests whether detectors can find reward hacking down to the exact command. https://huggingface.co/datasets/facebook/hackbench 21. Agent training data should cover tricky multi-step dependencies and keep only actions justified by what the agent could see. https://arxiv.org/abs/2610.33568 22. Agent training should teach the model to predict environment observations, not only which actions to take. https://arxiv.org/abs/2609.20715 23. An open model tried a rare unrequested upload in testing. Cohere cannot tell if it was planted or learned from agent logs. https://cohere.com/blog/concerning-behaviour-popular-open-weight-ai-model 24. Publishing every test run, including failures, lets anyone check how a model's scores were earned. https://huggingface.co/datasets/Hcompany/trajectories 25. Decontaminating math training data by deleting anything similar to benchmark questions throws away mostly good, unrelated items. https://arxiv.org/abs/2610.21229 26. Judge AI agents on their full trajectories, not just the final result, and check that the monitors themselves work. https://x.com/random_walker/status/2112509887456428208 27. A distillation dataset with no documented filtering, sources or rights is risky to use. https://huggingface.co/datasets/MoreThought/Fable-5.1-Max-Reasoning-Filtered-10000x 28. The command line can serve as a general workspace for many kinds of professional tasks, not only programming. https://huggingface.co/datasets/shunzou05/TerminalHorizon-Environment 29. Use rubrics first as hidden hints for a teacher model, then as the reward in RL. https://arxiv.org/abs/2610.02781 30. Midtraining on real code history and agent logs makes coding agents better, even with identical SFT afterward. https://arxiv.org/abs/2610.22019 31. The best midtraining checkpoint is the one that does best after post-training, not the one with top raw scores. https://x.com/cwolferesearch/status/2090080281248325744 32. How hard a task is depends on the task, the model and the agent setup together. https://huggingface.co/datasets/ai-and/termgrade-environments ### Recipe: Preference and reward-model data Status: Debated. Use when: Quality is a matter of comparison (helpfulness, tone, taste, safety judgment) and you need DPO pairs or a reward model where no program can check the answer. ## One Record ``` { "pair_id": "p-000981", "prompt": "", "prompt_source": "", "prompt_group": "", "chosen": { "text": "", "generator": "", "tokens": "" }, "rejected": { "text": "", "generator": "", "tokens": "" }, "on_policy": true, "label": { "method": "expert_human | crowd_human | ai_judge | delta_heuristic", "raters": [ "" ], "rater_credential": "", "agreement": "", "aspects": { "accuracy": "chosen", "safety": "tie" }, "authenticity_check": "pass" }, "split": "train | rm_heldout" } ``` ## Typical Sizes - Open DPO mix size: 354,192 curated preference instances in Tulu 3; the final DPO mixes used 271,409 (8B) and 334,302 (70B). [1] - Reward-model training pairs: About 10,000 response pairs in HelpSteer2, which its authors say trained a top RewardBench model in 2024. [2] - AI-feedback scale: Over 1 million GPT-4 ratings for 250k conversations in UltraFeedback. [3] - Expert rater pool in a recent domain pilot: 100 verified clinicians rating six frontier models (Prolific, company-reported). [4] - Generative reward-model sizes studied: Judges from 0.6B to 14B training policies from 0.6B to 8B; larger judges kept helping, including judges larger than the policy. [5] ## Filters, in Order 1. Draw chosen and rejected responses from many generators, and check that length or generator identity alone cannot predict the label.: Pairs from one fixed small-vs-large model pair let generator style predict the label; DPO on them underperformed multi-model data, and more generators partly fixed it. [6] 2. For distillation-style DPO, source rejects from smaller frozen models as well as the student.: Rejects from a 3B model improved a 14B student by 3.1 (code) and 1.9 (math) points over the student's own rejects. [7] 3. Include on-policy responses from the model being trained.: Tulu 3 reports on-policy data improved DPO performance. [1] 4. Use credentialed raters where errors need expertise.: Public raters chose a clinician-flagged reply 46% of the time vs 19% for clinicians (company pilot). [4] 5. Screen human raters for AI use and bots, and report the flagged rate.: Prolific reports behavior checks with 98.7% precision and in-question reminders cutting AI use 61% (vendor-reported). [8] 6. Consider token-level correction of the model's own outputs instead of writing answers from scratch.: onPanda cut median annotation time 52% in a small controlled study. [9] 7. Decontaminate prompts against evals before labeling.: Tulu 3 removed overlapping prompts with 8-gram matching; up to 11.3% of one source was removed. [1] ## Splits Split by prompt group so no prompt or near-duplicate crosses train and held-out, and make the build fail on overlap (hf-maziyarpanahi-agenttooldecisions-180k, hf-localllama-typed-decisions). Keep a held-out set of expert-labeled pairs, plus an out-of-distribution set with verifiable reasons for preference (the RewardBench design, url:https://arxiv.org/abs/2403.13787), to check reward models before using them in training. ## Validate Before Training - [ ] Style and length probe (pass when: A simple classifier on length, format or generator identity predicts the chosen side little better than chance.) [6] - [ ] Reward model or judge chosen by the policy it trains (pass when: A short pilot compares policies trained with each candidate on a held-out human-checked set.) [5] - [ ] AI labels vs expert labels (pass when: AI-judge preferences agree with expert raters on a held-out set at a level near expert-expert agreement.) [4] [10] - [ ] Overoptimization watch during RL (pass when: A held-out gold score (human or stronger judge) keeps rising with the proxy reward; stop when it flattens or falls.) [11] - [ ] Teacher ceiling for model-made labels (pass when: Scores above the labeling model's self-agreement are treated as copying its quirks, not as gains.) [12] ## Pitfalls - A fixed stronger-vs-weaker model pair is contested: delta learning matched Tulu 3 at 8B with 3B vs 1.5B pairs, but a 2026 study found fixed pairs teach generator style. [13] [6] - The most accurate judge on a static test is often not the best rewarder: answer-only judges gave higher peak policy Elo in 7 of 8 settings. [5] - Optimizing a reward model too hard lowers a gold reward model's score (a stand-in for human judgment in Gao et al.'s synthetic setup); the curve's coefficients scale smoothly with reward model size. [11] - AI judges can disagree sharply with each other on the same items (25% vs 74% flagged). [4] - Some teams now replace human preference labels with RL against simulated people; humans& reports gains for its tutor, but these are company self-reports and real-world learning gains are not yet measured. [14] ## Sources 1. Tulu 3: Pushing Frontiers in Open Language Model Post-Training (Lambert et al., 2024) https://arxiv.org/abs/2411.15124 2. HelpSteer2: Open-source dataset for training top-performing reward models (Wang et al., 2024) https://arxiv.org/abs/2406.08673 3. UltraFeedback: Boosting Language Models with Scaled AI Feedback (Cui et al., 2023) https://arxiv.org/abs/2310.01377 4. In medicine, what ordinary people prefer is not a reliable sign that an answer is safe. https://www.prolific.com/resources/what-s-new-edit-live-studies-run-frame-specific-video-evals-and-more 5. Choose an LLM judge by how good a model it helps train, not by its static benchmark score. https://arxiv.org/abs/2610.30765 6. If the preferred answer always comes from the same bigger model, training learns that model's style instead of real quality. https://arxiv.org/abs/2610.37058 7. In DPO, rejected answers work best when on-task but unlike the model's own answers. https://arxiv.org/abs/2609.38987 8. Human data can be screened for AI use by watching how people behave, not only by reading their text. https://www.prolific.com/resources/introducing-authenticity-checks-beta-ensure-genuine-human-responses-in-the-age-of-ai 9. Having people correct individual tokens in a model's own answers cheaply produces on-policy training data. https://arxiv.org/abs/2609.24983 10. LLM judges agreeing with each other does not mean they agree with human experts. https://arxiv.org/abs/2610.13303 11. Scaling Laws for Reward Model Overoptimization (Gao et al., 2022) https://arxiv.org/abs/2210.10760 12. When answer labels come from a model, scores above that model's agreement with itself mean learning its quirks. https://huggingface.co/datasets/LocalLLaMA/typed-decisions 13. The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gains (Geng et al., 2025) https://arxiv.org/abs/2507.06187 14. When RL uses simulated users, a better simulated user produces a better model. https://tutor.humansand.ai/blog/ 15. Labeled agent choices, such as whether to call a tool, suit small and fast classifier models. https://huggingface.co/datasets/MaziyarPanahi/AgentToolDecisions-180K ### Recipe: Held-out eval set Status: Emerging. Use when: You need a number you can trust for decisions: release gates, data-method comparisons, or checking a vendor's claim. ## One Record ``` { "item_id": "ev-0042", "split": "heldout_private | public_calibration", "source": "", "created_at": "2026-09-14", "canary": "", "env_ref": "", "reference_solution": "", "graders": [ { "type": "code", "path": "tests/test.sh", "image": "" }, { "type": "model", "judge": "", "prompt_version": "" } ], "should_not_twin": "ev-0043", "failure_mode": "", "irt": { "difficulty": "", "discrimination": "" }, "trials": 5, "metrics": [ "pass@1 with CI", "pass^5" ], "audit": { "failing_traces_read": true, "passing_traces_read": true, "verdict": "usable | usable_with_fixes | flawed" } } ``` ## Typical Sizes - Private or expert eval sets: Tens to a few hundred tasks: 60 (Surge sudo L7), 95 (FDE-Gym), 119 (SWE-bench Science), 240 (APEX-Agents 1.1, 80 per domain), 262 private test questions (Vals Immigration Bench). [1] [2] [3] [4] [5] - Trials per task: 5 (IBM pass^5), 10 attempts per problem (AIME in the reasoning-effort set), 20 (Microsoft ThinkingBox-Bench). [6] [7] [8] - Public calibration split: Artificial Analysis keeps a 300-question public set that correlates 0.99 with its private score (company-reported). [9] - Private share in a major index: 60% of Artificial Analysis Intelligence Index v5 uses private or held-out sets. [10] - Defect threshold for a benchmark verdict: Epoch rates a benchmark Flawed when defects could affect at least 20% of the full score; 9 of its first 15 were rated Flawed. [11] ## Filters, in Order 1. Read failing traces and label each failure as model, task, grader or harness bug; audit tasks every model fails first.: Failures often come from ill-specified questions, bad graders or harness design; Epoch found 23 false negatives in DeepSWE's 113 tasks. [12] [11] 2. Read passing traces for cheats and run a do-nothing agent; score cheated passes zero.: Preference Model found 44 of 65 Terminal-Bench 4 tasks passable by cheating; agentic benchmarks have counted empty responses as success. [13] [14] [15] [16] 3. Ablate inputs the task claims to need, and run a control where the input is known to matter.: SWE-bench Multimodal v2 scored 36.9% with images and 37.1% without, while the same ablation cut MMMU-Pro from 79.7% to 36.9%. [17] 4. Source private tasks from a different distribution than public sets, including tasks where the agent must discover the problem.: Mercor's CEO argues same-distribution held-outs generally will not stop overfitting (vendor that sells private evals); Surge grades raising unasked issues. [18] [19] 5. Keep tasks that separate models (item response theory) and cap tasks per failure mode.: Benchmarks built from one replicated failure swing from near zero to near 100% once it is fixed. [20] [21] [22] 6. Retire leaked and saturated items with a version bump.: Terminal-Bench 4.0 removed 8 tasks (2 with public solutions, 2 saturated) and fixed 19; AA-Omniscience dropped public and saturated questions. [23] [9] 7. Make held-out splits at least as hard as training and say so.: SmolDataEnvs made held-out splits 38 to 40% hard vs 14% in train. [24] 8. Run evals with live internet off and tamper-resistant transcript viewing.: Anthropic found real-world workarounds in evals and a misconfigured task led to unsanctioned actions in 9 of 43 runs; METR showed an agent could rewrite the transcript a reviewer sees. [25] [26] [27] ## Splits Keep a small public calibration split and a larger private split, check that the two track each other (Artificial Analysis reports 0.99 correlation, company-reported, x-artificialanlys-2116319506318635503), and enforce zero overlap with training task ids by script; Typed Decisions refuses to build if any case crosses splits (hf-localllama-typed-decisions). Decontaminate training data against evals with retrieval plus judge adjudication rather than a cosine threshold alone (arxiv-2610.21229). ## Validate Before Training - [ ] Reliability across trials (pass when: Report mean with a confidence interval and pass^k at a fixed k; IBM saw mean@5 77.4% vs pass^5 53.0%.) [6] [28] [29] - [ ] Corrected score after fixing task and grader bugs (pass when: Original and corrected scores are both published with a verdict (Preference Model: 62.8% original vs 89.2% fixed for one model).) [13] [11] - [ ] Judge agreement with experts on items where experts agree (pass when: Reported per eval (Vals reports 88.6%, vendor-reported).) [5] [30] - [ ] Paired statistics for model or data comparisons (pass when: Differences are reported with paired 95% intervals across seeds and items.) [31] [29] - [ ] Shared result schema with config context (pass when: Harness, provider, judge and prompt versions are recorded per run, e.g. in the Every Eval Ever schema.) [32] ## Pitfalls - Trusting leaderboard numbers you have not rerun; reruns at different context or reasoning budgets can differ (e.g. 91.88% vs a claimed 100%). [33] [12] - Private sets lose the outside readers who find bugs; Lambert says he is 50/50 on private vs open. [34] - Images were rated unnecessary for 403 of 480 SWE-bench Multimodal v2 tasks, so the score did not measure the claimed skill. [17] - Using model self-assessment or agent-written reports as the score; agents rated their own papers far above expert verdicts. [35] - Monitoring gaps: METR found OpenAI's misalignment monitor did not cover RL training runs at the start of its pilot. [36] ## Sources 1. Unit tests no longer separate top coding agents, so engineering judgment must be scored with expert rubrics. https://surgehq.ai/blog/sudo-l7 2. Finding requirements customers never state is the main hurdle for agents doing deployment work. https://huggingface.co/datasets/DeepLethe/FDE-Gym 3. Putting hidden tests in a separate container keeps them out of the agent's reach. https://huggingface.co/datasets/OpenMOSS-Team/SWE-bench-Science 4. Rubrics that reward including something invite agents to list many answers, so the judge must penalize that. https://mercor.com/blog/introducing-apex-agents-1-1/ 5. Expert rubrics catch models giving confident but outdated answers, especially on rules that changed recently. https://www.vals.ai/blogs/can-ai-get-immigration-right 6. An agent's average success rate can look high while it still fails many tasks on some runs, so measure passing every time. https://huggingface.co/blog/ibm-research/altk-evolve-consistency 7. This dataset shows how changing a model's reasoning effort changes results on identical tasks, with full trajectories. https://huggingface.co/datasets/KaiserWhoLearns/ReasiningEffortTrajec 8. Grading agents on the final database state and training only on medium-difficulty tasks improved pass@1. https://huggingface.co/blog/microsoft/thinkingbox-training 9. A small public question set can stand in for a private one, if you check that both give the same scores. https://x.com/ArtificialAnlys/status/2116319506318635503 10. The headline AI model rankings from Artificial Analysis now rest mostly on tests that labs cannot see or train on. https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5 11. Many widely used AI benchmarks have serious errors, so they need their own quality checks before teams rely on them. https://x.com/EpochAIResearch/status/2100704765332394255 12. When a model fails a benchmark question, the question itself is often broken, so check failures before trusting the score. https://x.com/HamelHusain/status/2114935968604299552 13. A popular agent benchmark had flaws in most tasks, and once fixed, a top model nearly maxes it out. https://x.com/chem_safety/status/2112212878132744674 14. Some passing scores on coding benchmarks come from reward hacking or weak graders, so audit passes per model. https://arxiv.org/abs/2609.34262 15. Passes that come from reward hacking, like looking up published answers, now earn zero in Artificial Analysis's coding index. https://x.com/ArtificialAnlys/status/2092406804424839199 16. Establishing Best Practices for Building Rigorous Agentic Benchmarks (Zhu et al., 2025) https://arxiv.org/abs/2507.02825 17. If removing images does not change scores, a test is not really measuring image understanding. https://x.com/dejavucoder/status/2112910513982292256 18. Model builders tune to public tasks, and held-out splits drawn from the same public sets generally won't fix it, he says. https://x.com/BrendanFoody/status/2113775443639960056 19. Good evals should hide the real problem the way actual client requests do, then check whether the model finds it. https://x.com/echen/status/2115189693159809472 20. Choose eval tasks that show which model is better, not just tasks that are hard. https://x.com/cwolferesearch/status/2115121894328603063 21. Benchmarks that repeat one known model failure stop being useful the moment that failure is fixed. https://x.com/MTSlive/status/2115223600391418263 22. Hidden bias shows up when a model faces equal pressure for each candidate, not in its final picks alone. https://allenai.org/blog/fairstress 23. Retire test tasks whose answers have leaked or that models already ace, and release breaking changes as new versions. https://github.com/harbor-framework/terminal-bench/releases/tag/v4.0.0 24. Check answers automatically where possible, and make test sets harder than training sets, saying so openly. https://huggingface.co/datasets/FineEnvs/SmolDataEnvs 25. Rewarding persistence in training can teach AI agents to get around restrictions in the real world. https://www.anthropic.com/research/investigating-unintended-model-actions 26. A badly set up test can push an AI agent into harmful real-world actions, so tasks need checking before they run. https://www.anthropic.com/research/alignment-assessment-ukaisi 27. Tools for viewing AI agent logs can themselves be attacked, so agent output must be treated as untrusted. https://metr.org/blog/2026-10-06-ai-systems-could-cover-up-misbehavior/ 28. tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (Yao et al., 2024) https://arxiv.org/abs/2406.12045 29. Adding Error Bars to Evals (Miller, 2024) https://arxiv.org/abs/2411.00640 30. Human agreement is the anchor for judging open-ended work, and LLM judges are a calibrated stand-in, not ground truth. https://x.com/cwolferesearch/status/2112915508148232571 31. Clever ways to pick or weight training problems did not beat simple random sampling in this study. https://arxiv.org/abs/2609.06107 32. The UK AI Security Institute is publishing model eval results with full settings, in a shared format, so others can compare them. https://huggingface.co/blog/evaleval-aisi 33. When rival vendors publish test scores, rerun them yourself with the same settings before trusting the comparison. https://x.com/danielhanchen/status/2116162363313525191 34. Keeping a test private stops labs from training on it, but it also loses the outside reviewers who find its mistakes. https://x.com/natolambert/status/2115437422146130032 35. On open-ended work, an agent's opinion of its own output can be far from what experts think. https://x.com/pkirgis/status/2112617994496991379 36. Safety monitoring must cover training runs too, and agents should not control when they get checked. https://metr.org/blog/2026-10-22-red-teaming-openai-agent-monitoring/ 37. When answer labels come from a model, scores above that model's agreement with itself mean learning its quirks. https://huggingface.co/datasets/LocalLLaMA/typed-decisions 38. Decontaminating math training data by deleting anything similar to benchmark questions throws away mostly good, unrelated items. https://arxiv.org/abs/2610.21229 ### Recipe: Teacher distillation data Status: Emerging. Use when: You want to move skills from a stronger model, or from the same model given privileged context, into a smaller or cheaper student, often as a warm start before RL. ## One Record ``` { "example_id": "d-004512", "prompt": "", "student_input": "", "teacher": { "model": "", "tokenizer_shared_with_student": true, "terms_allow_training": true, "privileged_context": "" }, "mode": "on_policy_reverse_kl | forward_kl | offpolicy_sft", "slice_goal": "skill | fact", "student_sample": "", "teacher_logprobs_ref": "", "filter": { "rubric_all_pass": true, "verified": true, "best_of": 8 }, "lengths": { "student_tokens": "", "teacher_tokens": "" }, "provenance": { "teacher_date": "2026-10-01", "purpose": "warm start before RL" } } ``` ## Typical Sizes - On-policy distillation run: About 77K prompts x 4 samples over about 150 steps took a model from 60% to 70% on AIME'24; default batch 64 prompts x 4 samples (Thinking Machines). [1] - Compute vs alternatives: About 9 times lower cost than off-policy SFT to reach the same AIME'24 score when the SFT data already exists, about 30 times counting teacher sampling; and 1,800 vs 17,920 GPU-hours vs RL in the Qwen3 comparison Thinking Machines cites (author-reported). [1] - Off-policy teacher-trace sets: 10,000 traces (about 500M tokens) to 414,890 samples: MoreThought Fable distill, Nemotron-Math-Proofs-v3-SFT. [2] [3] - Candidates per prompt before keeping one: 8 (Samaya's eight prompted strategies, company-reported; NVIDIA finance's eight trajectories). [4] [5] - Task-targeted distillation sets: 538 rubric-filtered tasks (AfterQuery, company-reported); 3,365 self-context examples plus 972 reflections (Bespoke Labs, company-reported); 4,500 prompts (Meta rubric-privileged OPD). [6] [7] [8] ## Filters, in Order 1. Check whether a warm start is needed: measure any-correct-in-16 and mixed-reward-group share on the RL prompts first.: Teacher SFT helped 0.6B and 1.7B models but an 8B model that already solved 59.4% of prompts at least once did 6.4 points better with RL alone (company research). [9] 2. Pick the objective per slice: reverse-KL on-policy distillation for skills, forward KL or SFT for facts.: Reverse KL lifted unseen composition from 1.8% to 76.8% but fact recall reached 10.95% vs 54.10% with forward KL. [10] 3. Give the teacher privileged context (such as the rubric) but keep it out of student inputs and saved text.: Rubric-privileged on-policy distillation then RL beat rubric-conditioned SFT then RL: 0.673 vs 0.607 on HealthBench for Qwen2.5-7B. [8] 4. Filter teacher outputs with rubrics or verifiers; keep only outputs that pass every criterion.: AfterQuery kept only all-criteria passes and reports gains on a held-out set (company-reported). [6] 5. Select traces for diverse reasoning routes.: Route-diverse SFT raised held-out pass@8 after RL by 16.9 points vs similar-route traces from the same pool. [11] 6. Use teachers whose terms allow training, and log teacher, date and purpose per batch.: Anthropic says it attributed over 16 million exchanges through about 24,000 fraudulent accounts to distillation campaigns by three labs. [12] 7. Track and report output length per arm.: Distilled outputs grew about 74 to 80% longer in AfterQuery's run, and the strongest teacher-free result had longer answers than its baseline. [6] [13] ## Splits Evaluate on benchmarks and environments the teacher never generated data for, and add a same-teacher control on other content so teacher quality is not credited to the data (in GameSmith a non-game distillation control reached 60.8 vs 62.5, arxiv-2610.13995). Decontaminate teacher prompts against evals before generation (url:https://arxiv.org/abs/2411.15124). ## Validate Before Training - [ ] Teacher-free arm (pass when: Distillation beats a teacher-free variant at matched steps and samples, with lengths reported (OPSA beat OPD 48.85 vs 32.08 on AIME24, lengths not matched).) [13] - [ ] Same-teacher control (pass when: The gain over a same-teacher, other-content control is reported, not only over the base model.) [14] - [ ] Warm-start checkpoint chosen by post-RL score (pass when: Each candidate has a score after a fixed RL budget; pre-RL accuracy is not used to choose.) [9] [8] - [ ] Entropy and mixed-group share through the warm start (pass when: Chosen checkpoint sits before any sharp drop.) [9] [8] - [ ] Claimed-compliance regrade after RL (pass when: Substance-only regrade stays within tolerance of training reward.) [8] - [ ] Per-benchmark results, not just averages (pass when: Methods that tie on average (within 1.4 points) are also compared per benchmark (gaps up to 8.6).) [15] ## Pitfalls - Expecting reverse-KL distillation to teach new facts. [10] - About 30.6% of a 4B teacher's token advantages pointed the wrong way in one study, yet a fixed negative advantage matched teacher signal; part of the gain may not need a teacher. [13] - Students are capped by the teacher; when gold labels come from a model, scores above its self-agreement (0.735 in Typed Decisions) mean copying quirks. [16] - Undocumented distillation sets (no filters, sources or rights) are hard to use for buyers who need provenance. [2] - Off-policy SFT warm starts can teach claimed compliance and lower later RL scores when run long. [8] ## Sources 1. On-Policy Distillation (Thinking Machines Lab, Kevin Lu et al., 2025-10-27) https://thinkingmachines.ai/blog/on-policy-distillation/ 2. A distillation dataset with no documented filtering, sources or rights is risky to use. https://huggingface.co/datasets/MoreThought/Fable-5.1-Max-Reasoning-Filtered-10000x 3. Use distillation from a stronger model to teach writing, improving and checking math proofs together. https://huggingface.co/datasets/nvidia/Nemotron-Math-Proofs-v3-SFT 4. SFT on graded teacher examples, inside the agent's real tools, produced most of one finance agent's improvement. https://samaya.ai/blog/rl-post-trained-agentic-model 5. NVIDIA released finance agent trajectories for SFT, keeping one best answer per question. https://huggingface.co/datasets/nvidia/Nemotron-SFT-Agentic-Finance-v1 6. Distillation works cleanly when only teacher answers passing every rubric check are kept. https://www.afterquery.com/blog/xtoken-distillation-gdp-pdf 7. Training a model on its own successful moves, with hints removed, plus self-written lessons, beat training on curated teacher examples. https://www.bespokelabs.ai/blog/teaching-nvidias-nemotron-ml-engineering 8. Use rubrics first as hidden hints for a teacher model, then as the reward in RL. https://arxiv.org/abs/2610.02781 9. Use an SFT warm-up on teacher examples only when the model cannot yet solve enough tasks to learn from rewards. https://labs.baseten.co/articles/when-does-distillation-help-reinforcement-learning 10. On-policy distillation passes on reasoning skills but almost no facts. https://arxiv.org/abs/2610.09639 11. Picking SFT examples that solve problems in varied ways helps later RL. https://arxiv.org/abs/2609.33780 12. Detecting and preventing distillation attacks (Anthropic, 2026-02-23) https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks 13. Gains from on-policy distillation come mostly from discouraging unlikely tokens, not teacher knowledge. https://arxiv.org/abs/2608.31046 14. More game types help, but most of the gain comes from distillation, not the games. https://arxiv.org/abs/2610.13995 15. Merging weights, mixed-data RL and multi-teacher distillation average about the same for combining specialists. https://arxiv.org/abs/2608.27409 16. When answer labels come from a model, scores above that model's agreement with itself mean learning its quirks. https://huggingface.co/datasets/LocalLLaMA/typed-decisions ## Implementation Guides ### Build a Frontier-Calibrated RL Task Set A task pool where every task sits at the current policy's edge: measured pass rates, solvable hard tasks with fading hints, new problems grown from seeds, and scheduled refreshes. 1. Fix the Policy, Harness and Rollout Budget (done when: A versioned config records checkpoint hash, harness version, sampling settings and timeouts, and every later step reads from it.) 2. Measure Pass Rates and Separate System Errors (done when: Every task has passes, fails, system errors, N and an estimate stored; no task's rate counts system errors as fails.) 3. Bucket Tasks and Report the Zero-Signal Share (done when: A report lists counts per bucket, the zero-advantage share, and the chosen band with its source.) 4. Verify Always-Fail Tasks Are Solvable (done when: Each kept always-fail task has a passing reference run and either a stronger-model solve or a human solve on record.) 5. Add a Fading-Hint Curriculum for Hard Tasks (done when: Each hinted task logs its current hint fraction and pass rate, and hint fractions fall over training.) 6. Grow New Problems From Hard Seeds (done when: Every generated task has a parent seed id, a passing reference run, and a measured pass rate inside the keep band.) 7. Feed Solver Failures Back Into Generation (done when: Generator prompts or inputs include recent failure transcripts, and generated-task logs record which failures prompted them.) 8. Refresh the Pool During Training (done when: A scheduled job re-measures tasks every K steps and writes a pool diff: retired, added, hint changes. The refresh job stores reward-variation metrics and refresh triggers.) 9. Compare Against a Uniform-Sampling Baseline (done when: A results table shows the calibrated pool against uniform sampling at matched rollouts, with several seeds per arm.) 10. Store Calibration Metadata With Every Task (done when: Every task record validates against a schema with calibration fields, and no field lacks a checkpoint id.) ### Harden an RL Environment Against Reward Hacking An environment with leak paths scrubbed, tests out of reach, network exits closed, negative solutions that must score zero, measured cheat resistance and an outside red team report. 1. Inventory Every Leak Path (done when: A written leak inventory exists per environment family, with each path marked scrubbed, blocked or accepted with a reason.) 2. Scrub Leaks as a Versioned Image Step (done when: The image build includes a named, versioned cleanup stage, and a post-build scan returns no leaks on every task image.) 3. Move Tests and Grading Out of Reach (done when: An agent shell inside the task cannot list, read or write any test, solution or grader file, verified by a scripted probe.) 4. Close Every Network Exit (done when: From inside a task, scripted attempts at DNS, HTTP, model-API fetch and shared-cache writes all fail and raise alerts.) 5. Write Negative Solutions That Must Score Zero (done when: Every task ships multiple negatives, all score zero, and the reference solution scores one on repeated runs.) 6. Run Scripted and Model Cheating Probes (done when: A probe report lists attempts per task and zero successful cheats on admitted tasks.) 7. Run a Hacker-Fixer Loop on Each Verifier (done when: Each verifier has a stored exploit corpus that replays to zero, and honest solve rate is within your set tolerance of the original.) 8. Monitor Rollouts and Make Kill Switches Fire (done when: Monitor precision and recall are measured on a labeled set, and a drill shows an alert stops the run automatically.) 9. Simulate Hacking Onset Before a Big Run (done when: Pilot runs or a simulation report show detected hacking over steps for the chosen curriculum, reviewed before the full run.) 10. Commission an Outside Red Team and Report Incidents (done when: A signed outside report exists, every finding has a fix and a replay test, and incidents use the disclosure fields.) ### Build and Validate a Grader or Rubric Reward A reward function that passes correct work, fails wrong work and side effects, resists padding, and has a judge measured against human labels rather than other judges. 1. Choose the Cheapest Grader That Is Valid (done when: Each task or rubric item names its grader type, and no LLM judge scores anything a deterministic check could score.) 2. Write the Gold First and Prove It Scores Full Marks (done when: Gold scores full marks on every repeated run; excluded tasks are listed with reasons.) 3. Check the Whole End State and Add Guardrails (done when: Every task has at least one guardrail or count assertion, and the reference passes all of them.) 4. Test Both Directions: Wrong Must Fail, Right Must Pass (done when: Per task, all known-wrong answers score 0 and all collected alternative correct answers score 1.) 5. Build Rubric Rewards With Gates and a Precision Term (done when: The scoring function has all-or-nothing groups, a penalty for wrong or unneeded content, and rubric text absent from policy prompts.) 6. Red-Team the Rubric Before Training (done when: An attack set per task family scores below a set threshold, and every fixed rubric item has a regression attack.) 7. Calibrate the Judge Against Human Labels (done when: Judge kappa and fail-class recall on held-out human labels are reported next to human-human kappa.) 8. Use a Judge Panel That Masks Disagreement (done when: Training logs show the share of rollouts masked for judge disagreement, and each panel member has a calibration report.) 9. Select the Judge by the Policy It Trains (done when: A pilot table compares policies trained with each candidate judge on a held-out human-checked set.) 10. Cross-Check, Freeze and Version the Grader (done when: Second-grader agreement is reported, disagreements are triaged, and every reward record carries a grader version.) ### Train and Validate a User Simulator for Agent RL A simulated user that follows private task specs, behaves like real people rather than a helpful assistant, and is checked against real-user outcomes before it shapes agent RL. 1. Measure the Real-User Baseline (done when: A baseline table reports real-user success rate, turns per episode and disclosure timing for a fixed agent on a task sample.) 2. Write Private User Specs per Task (done when: Every task has a spec with goal, facts and disclosure rules, stored in a field excluded from agent prompts.) 3. Annotate Real Behavior Patterns (done when: A behavior taxonomy with measured frequencies from real data exists, with at least a few hundred tagged turns.) 4. Choose or Train the Simulator (done when: A simulator checkpoint or prompt is versioned, with its training data sources and licenses recorded.) 5. Validate Fidelity, Realism and Calibration (done when: A validation report gives spec-violation rate, judge-fooled rate, success gap against real users and behavior frequency match.) 6. Build the Environment Around the Simulator (done when: Rewards come from end-state graders; simulator turns, seeds and spec ids are logged for every episode.) 7. Test Generalization Across Simulators (done when: Eval results are reported per held-out simulator, and gains hold on the majority of them.) 8. Watch for Simulator Exploits During RL (done when: Training dashboards show spec-violation rate per step, and sampled high-reward transcripts are reviewed every evaluation interval.) 9. Revalidate Against Fresh Real Data (done when: A dated revalidation report exists for each simulator version, and product claims cite real-user results.) ### Run On-Policy Distillation Before RL A warm-start stage where the student learns on its own samples, scored by a teacher or a fixed signal, chosen and checked by how well RL goes afterward. 1. Decide Whether You Need a Warm Start (done when: A pre-RL report gives any-correct-in-16 and mixed-group share, and a written decision to warm start or go straight to RL.) 2. Decide What Must Transfer: Skills or Facts (done when: The plan states, per data slice, whether the target is skill or fact and which objective each slice uses.) 3. Set Up the Teacher and Any Privileged Context (done when: A dry run shows teacher log-probabilities aligned to student tokens, and the student input contains no privileged fields.) 4. Implement the On-Policy Loop (done when: Training runs end to end, per-token advantages are logged, and reverse KL to the teacher falls on a fixed probe set.) 5. Run a Teacher-Free Baseline (done when: Results include a teacher-free arm at the same steps and samples, with response lengths reported for every arm.) 6. Protect Entropy and Diversity for RL (done when: Entropy and mixed-group share are plotted across the warm start, and the chosen checkpoint is before any sharp drop.) 7. Hand Off to RL and Judge by Post-RL Results (done when: Each candidate warm start has a post-RL score at a fixed RL budget, and selection is based on that number.) 8. Audit for Claimed Compliance (done when: A regrade of sampled outputs is reported next to the training reward, and the gap is within a stated tolerance.) 9. Compare Arms at Matched RL Budget (done when: A table shows all arms at matched RL tokens with seeds, per-benchmark scores and averages.) ### Run a Fair Data Experiment A controlled comparison that can credit a data method: tuned baselines, controls that isolate the ingredient, matched compute, several seeds, balanced held-out evals and full reporting. 1. Write the Claim and Plan Before Training (done when: A committed plan file predates the first training run and names metric, evals, arms, seeds, compute and analysis.) 2. Tune the Baseline as Hard as the Method (done when: Baseline and method have logged hyperparameter sweeps of similar size, and the baseline passes an overfit-small-pool sanity test.) 3. Add Controls That Isolate the Ingredient (done when: Each ingredient in the claim has a control arm that differs only in that ingredient.) 4. Match Compute and Training Settings (done when: A compute table lists rollouts, tokens, steps, batch size and synthesis tokens per arm, all within a stated tolerance.) 5. Run Several Seeds and Use Paired Statistics (done when: Every arm has at least three seeds, and the headline difference has a paired 95% interval.) 6. Evaluate on Balanced, Held-Out, Clean Suites (done when: Eval list, decontamination report and harness and judge versions are identical across arms.) 7. Measure More Than the Mean (done when: The results table has pass@1, pass@k, pass^k and audited hack rate per arm with intervals.) 8. Report Everything, Including Nulls (done when: The write-up includes the full arm table, compute, seeds, intervals, rerun status, limits and configs.) ### Audit and Filter Agent Trajectories for SFT An agent SFT set where every trajectory has provenance, a re-verified outcome, a hack check, grounded actions and masked bad turns, so the model learns work, not shortcuts. 1. Standardize the Schema and Provenance (done when: Every record validates against the schema; none lacks generator, harness, license or environment reference.) 2. Separate Infrastructure Failures From Agent Failures (done when: Infra-error trajectories carry resolved equal to minus 1 and are excluded from success and failure counts.) 3. Re-Verify Outcomes With Stronger Checks (done when: Every kept success passes the original verifier, the extra generated tests and the end-state checks on replay.) 4. Detect Reward Hacks With Rules and a Monitor (done when: Detector precision and recall on a labeled set are reported, and every flagged trajectory is removed or masked.) 5. Hand-Audit a Stratified Sample of Passes (done when: An audit log covers every stratum with a recorded sample size, and each new finding has a matching rule.) 6. Drop Actions the Visible History Does Not Justify (done when: A grounding checker runs on every tool call, and ungrounded calls are masked or the trajectory is dropped.) 7. Mask Bad Turns Instead of Deleting Runs (done when: Each assistant turn has an explicit train flag, and masked turns stay in context.) 8. Scrub Unwanted Behavior and Sensitive Data (done when: A scan for exfiltration attempts, credentials and personal data runs on all kept trajectories with zero unresolved hits.) 9. Select for Diversity, Decontaminate and Document (done when: A data card lists counts in and out at every filter, harness mix, decontamination method and license per source.) ### Build a Private Held-Out Eval and Audit Public Benchmarks An audit method for public benchmarks plus a private, differently sourced held-out eval with validated graders, discriminative tasks, reliability metrics and outside review. 1. Run the Public Benchmark Yourself With a Fixed Config (done when: A run config and full transcripts exist for every task, with trials per task recorded.) 2. Read Failing Traces and Classify Each Failure (done when: Every failing task has a label, and the share of score affected by bugs is computed.) 3. Read Passing Traces for Cheats (done when: A sample of passes per model is reviewed, cheated passes are zeroed, and a do-nothing agent score is recorded.) 4. Ablate Inputs the Task Claims to Need (done when: An ablation table shows scores with and without each claimed input and leak path, plus a control.) 5. Publish a Corrected Score and Verdict (done when: Each audited benchmark has an original score, a corrected score, a task-level fix list and a verdict.) 6. Source Private Tasks From a Different Distribution (done when: Each private task has a recorded source outside public benchmarks, and none is a paraphrase of a public item.) 7. Build Each Item With a Validated Grader (done when: Every item has reference, validated grader, twin, date and canary, and a script confirms zero overlap with training task ids.) 8. Keep Tasks That Separate Models (done when: Each task has difficulty and discrimination estimates, and no single failure mode exceeds a set share of the set.) 9. Run With Trials and Report Reliability (done when: Each report shows mean with interval, pass^k at a fixed k, and full config versions.) 10. Maintain the Eval and Invite Outside Audits (done when: A dated outside review, a per-release trace audit and a changelog of retired and added tasks exist.) ## Last 30 Days, Newest First - 2026-11-02 [Release; Dataset card] A CC0 tabular index catalogs public sources with licenses and per-source metadata. (Battam). https://huggingface.co/datasets/Battam/agentic-env-index - 2026-11-01 [Release; Dataset card] A large, gated dataset of real multi-agent trajectories with screenshots enables credible agent evals and behavior analysis. (AI Digest). https://huggingface.co/datasets/aidigestorg/ai-village - 2026-11-01 [Industry; X post] An anonymous black hat group offered a large sum to post-train open-weight models on exploit data. (Jun Song). https://x.com/jun_song/status/2116717099402096682 - 2026-11-01 [Release; X post] Routing a user-facing session to parallel isolated workers reportedly cut p95 completion time in DeepSeek’s Super-agent prototype. (Chubby♨️ (kimmonismus)). Result: p95 completion time 29.5 min -> 15.7 min (-13.8 min), vs single session using subagents. https://x.com/kimmonismus/status/2116905923109089285 - 2026-11-01 [Release; X post] A large DNA and text dataset with SFT and RL samples was released for training biology language models. (Loubna Ben Allal). https://x.com/LoubnaBenAllal1/status/2116785229159673914 - 2026-11-01 [Release; Dataset card] A CC0-licensed Hugging Face dataset lists 1K to 10K prompts. (fka). https://huggingface.co/datasets/fka/prompts.chat - 2026-11-01 [Legal and Policy; Company] House Judiciary Democrats challenge DOJ’s fair-use defense for AI training, spotlighting licensing risk and stage-by-stage boundaries. (Representatives Jerrold Nadler, Jamie Raskin and Hank Johnson). Deal: lawsuit . https://www.remio.ai/post/openai-copyright-dispute-nadler-raskin-and-johnson-challenge-dojs-ai-training-ar - 2026-11-01 [Opinion; X post] For post-training, proprietary data and processes are the competitive moat, not the base model. (Sridhar Ramaswamy). https://x.com/RamaswmySridhar/status/2116881230537146617 - 2026-11-01 [Release; Dataset card] QC index and aligned instructions support curation and auditing of image edits. (UCSC-VLAA). https://huggingface.co/datasets/UCSC-VLAA/GPT-Image-Edit-1.5M - 2026-11-01 [Release; Dataset card] A 5M-example synthetic agent dataset includes plans, tool traces and verification, with tool traces explicitly marked simulated. (Lelonthecodeur). https://huggingface.co/datasets/Lelonthecodeur/agent-task-management-dataset - 2026-11-01 [Release; Dataset card] An audited enterprise procurement RL environment and kit enables reproducible agent testing with shock scenarios and a passive counterfactual baseline. (Ahmed Mohamed). Result: terminal reward 0.2640 -> 0.4717 (+0.2077), vs Passive NO_ACTION counterfactual replay. https://huggingface.co/datasets/ahmed225005/ERP28-Strategic-Sourcing-RL-Evaluation-Kit - 2026-11-01 [Release; X post] Together offers a post-training loop that keeps rollout generation and training aligned while letting users bring their own rewards and verifiers. (Together AI). https://x.com/togethercompute/status/2116954280544329898 - 2026-11-01 [Release; Dataset card] A single schema packages 1.6M real human chats across five languages. (PetraAIi). https://huggingface.co/datasets/PetraAIi/human-chats-worldwide - 2026-11-01 [Release; Dataset card] Re-annotated MS MARCO with LLM search and grading yields cleaner positives and stricter dev qrels that improve retriever training. (Shroedingercat). https://huggingface.co/datasets/Shroedingercat/doubting-ms-marco - 2026-11-01 [Release; Dataset card] An Apache-2.0 dataset pairs Ghanaian search queries with Gemini-generated spoken responses and aligned transcripts for audio QA and conversational use. (Ghana Open AI Community). https://huggingface.co/datasets/ghanaopenai/ghana-search-queries-responses-audio - 2026-11-01 [Release; Dataset card] A fixed-protocol video set documents 1,800 DynamicVLA DOM validation trials for reproducible agent evaluation. (bathrlor). https://huggingface.co/datasets/bathrlor/DynamicVLA-DOM-Validation-Videos - 2026-11-01 [Opinion; Press] Buyers should demand explicit, uncapped IP indemnity that covers training data, model weights, and outputs in AI agent contracts. (Dave Barr). https://startupfortune.com/how-to-negotiate-an-ai-agent-training-data-indemnity-clause/ - 2026-11-01 [Release; Dataset card] A public release of containerized SWE environments and BM25 corpora to reproduce and train Octopus multi-agent RL. (Tianming Sha). Result: BrowseComp-Plus score -> 49.6% (+17.5 pts), vs 143K single-agent GRPO. Comparison: No clear control. https://huggingface.co/datasets/Tommysha/octopus-train - 2026-11-01 [Release; Dataset card] A signed, timestamped agent safety benchmark provides checkable rewards and evaluations for agent training. (CrawlCheck). https://huggingface.co/datasets/crawlcheck/agent-safety-benchmark - 2026-11-01 [Release; Dataset card] A full reproduction-ready benchmark for CAN-based GPS reconstruction with driver-disjoint folds and failure-aware scoring is released. (Anonymous authors). https://huggingface.co/datasets/CVPRICCVICML/GPS_Reconstruction - 2026-10-31 [Legal and Policy; Press] Website owners may soon get one standard way to refuse AI training, separate from blocking search crawlers. (Ana Maria Constantin, TNW). https://thenextweb.com/news/ai-training-opt-out-robots-txt-google-openai-microsoft-anthropic - 2026-10-31 [Research; X post] Training data can pass a line-by-line safety check and still teach a model to misbehave, so test what the data actually teaches. (Anthropic). https://x.com/AnthropicAI/status/2116349211549843666 - 2026-10-31 [Release; X post] A small public question set can stand in for a private one, if you check that both give the same scores. (Artificial Analysis). https://x.com/ArtificialAnlys/status/2116319506318635503 - 2026-10-31 [Industry; X post] Businesses whitelist Instinct and expose private APIs to support Instinct traffic under acceptable-use limits. (Noah Shinn). Deal: partnership . https://x.com/noahrshinn/status/2116626173212123177 - 2026-10-31 [Research; X post] Route tasks to clustered memory modules instead of one global history to raise reward and cut cost. (Michael Philip, Mahesh Balakrishnan, David Geraghty, David Mao, Vidhya Venkat, Ilya Mironov). Result: normalized reward Single global ICL history -> Routed Continual Learning (+15% to +30% of maximum normalized reward), vs Global ICL, matched rollouts. Comparison: Matched test. https://x.com/dair_ai/status/2116548002617917603 - 2026-10-31 [Research; X post] Gemini 4 Carbon scores 53 FCI and 54 pass rate as long-horizon evaluation is improved. (Wenhao Chai). https://x.com/wenhaocha1/status/2116412830610235564 - 2026-10-31 [Research; X post] A runtime decision gate that verifies tool calls and detects premature termination can boost agent reliability without changing models or tools. (elvis (@omarsar0)). Result: three-run success 39.5% -> 80.7% (+41.2 pts), vs agent without gate. Comparison: Matched test. https://x.com/omarsar0/status/2116547247018229999 - 2026-10-31 [Industry; Press] Enterprise buyers are rejecting default data pooling and demanding clear opt-in, reciprocity, and exit terms for shared AI training and enrichment. (Nur Hamzah). https://telset.id/news/ai/hubspot-batalkan-pooling-data-ai-pelanggan-menolak-berbagi-data - 2026-10-31 [Research; X post] Evaluate novelty claim-by-claim to avoid over-rejecting viable research ideas in AI scientist pipelines. (Rohan Paul). Result: False rejection rate on later-published ideas 96.0% -> 1.5% (-94.5 pts), vs a deployed check. Comparison: No clear control. https://x.com/rohanpaul_ai/status/2116621467219816915 - 2026-10-31 [Industry; X post] Cartwheel’s cofounder reports Asmov-1 nearing completion with 0.007% eval error and says open-source release should be next week including RP1. (Andrew Carr). https://x.com/andrew_n_carr/status/2116618273014124935 - 2026-10-31 [Release; X post] An open benchmark tests coding agents on Java framework migrations with deployment behavior verification, not just compilation. (Kaggle). https://x.com/kaggle/status/2116307166541860999 - 2026-10-31 [Research; X post] A purpose-built per-second activity grader can triage egocentric factory video more accurately than general multimodal models. (Humyn Labs). https://x.com/humynlabs/status/2116397726325985284 - 2026-10-31 [Industry; Press] Dimensional hired hourly operators to walk robot dogs in San Francisco for AI training. (Shubhi Mishra). https://www.moneycontrol.com/news/trends/san-francisco-startup-hires-robot-dog-walkers-for-ai-training-offers-rs-1-900-per-hour-14065006.html - 2026-10-30 [Release; Dataset card] A public index tracks listed hourly pay for AI training roles across platforms, with monthly medians and weekly mirrored editions. (AI Work Central). https://huggingface.co/datasets/AIWorkCentral/ai-training-pay-index - 2026-10-30 [Research; Company] When RL uses simulated users, a better simulated user produces a better model. (humans&). Result: TutorMoments overall score, plain prompt 0.635 -> 0.754 (+0.119), vs same tutor training with GLM-5.3 as simulated learner. Comparison: Matched test. https://tutor.humansand.ai/blog/ - 2026-10-30 [Release; X post] Even hard tests written by experts can need a year of cleanup and a re-release. (Artificial Analysis). https://x.com/ArtificialAnlys/status/2115989388689813945 - 2026-10-30 [Research; X post] In RL, a learned critic rarely beats spending the same compute on more rollouts. (Base Labs). https://x.com/baselabs/status/2116278798463492166 - 2026-10-30 [Opinion; X post] In Lambert's opinion, the mismatch between the inference engine that generates rollouts and the trainer is now the top research focus. (Nathan Lambert). https://x.com/natolambert/status/2116271281351881120 - 2026-10-30 [Opinion; X post] Fine-tune on all your data first, then finish by slowly lowering the learning rate while training only on your best subset. (kalomaze). https://x.com/kalomaze/status/2116206274333553068 - 2026-10-30 [Research; X post] When rival vendors publish test scores, rerun them yourself with the same settings before trusting the comparison. (Daniel Han). https://x.com/danielhanchen/status/2116162363313525191 - 2026-10-30 [Industry; X post] Google teased a world model that generates interactive simulations for training robots and AI agents before real-world deployment. (Google). https://x.com/Google/status/2116237215026032872 - 2026-10-30 [Opinion; X post] Train coding agents to regularize abstractions and equip them with a standard library. (Chris Lattner). https://x.com/clattner_llvm/status/2116268356160667775 - 2026-10-30 [Research; X post] Evolving the rubric with the harness keeps the critic aligned with current weaknesses. (DAIR.AI). https://x.com/dair_ai/status/2115987068761628740 - 2026-10-30 [Research; X post] Use human-guided benchmarks and RL with human simulators and rubrics to align autoresearch agents. (Jason Weston). https://x.com/jaseweston/status/2115992445930311714 - 2026-10-30 [Research; X post] Unify pretraining, SFT and RL as likelihood training, mixing static and RL data in one trainer. (Russ Salakhutdinov). https://x.com/rsalakhu/status/2116238535640465879 - 2026-10-30 [Release; X post] Micro1’s Dual-Arm Sim found frontier models rarely completed robotic surgery tasks and their self-reports were consistently wrong. (micro1). https://x.com/micro1_ai/status/2116236718957314116 - 2026-10-30 [Research; X post] Use scaling laws to decide compute budgets for synthetic rewrites and whether rewritten data can outperform the teacher. (Percy Liang). https://x.com/percyliang/status/2116236605262602604 - 2026-10-30 [Research; X post] Agentic evaluations with strict domain verifiers and refined MCP tool interfaces improved Siemens’ chip-design agent reliability and efficiency in NeMo Gym. (Joshua Mabry). Result: composite verification score ~55% -> ~95% (+40 pts), vs before harness optimization, same model and tasks. Comparison: Matched test. https://x.com/NVIDIAAI/status/2116236342653026706 - 2026-10-30 [Research; X post] A 4B decision model trained on 117k examples generalizes across tasks but fails sarcasm detection. (alphaXiv). https://x.com/askalphaxiv/status/2116199951835582507 - 2026-10-30 [Release; X post] A strong reranker reduces agent token use and improves retrieval, while synthetic fine-tuning adapts retrievers to domain data efficiently. (NVIDIA). Result: nDCG@10 on held-out chart and infographic retrieval 37.34 -> 44.12 (+6.78 pts (+18% relative)), vs vs base model before fine-tuning. Comparison: Vs. starting model. https://x.com/NVIDIAAI/status/2115962853463396395 - 2026-10-30 [Industry; X post] APEX-Agents reports Gemini 4 Carbon leading with 84.2% Pass@1 and 90.4% mean across multi-step tasks in consulting, banking, and corporate law. (Mercor). https://x.com/mercor/status/2116226723968872959 - 2026-10-30 [Research; X post] FrontierCode 2.0 shows Gemini 4 Carbon at 40% with costs per task, placing it near SOTA. (Lisan al Gaib). https://x.com/scaling01/status/2115983524041822606 - 2026-10-30 [Research; Press] Gradient attribution can pre-filter SFT data to reduce jailbreak risk without hurting capabilities. (CMU, University of Washington, and Meta AI researchers). Result: GCG jailbreak success on HarmBench 10.2% -> 3.8% (-6.4 pts), vs Training on full Tulu 3 versus SafeTulu filtered data. Comparison: No clear control. https://www.techtimes.com/articles/329767/20261030/meta-ai-cmu-researchers-cut-ai-jailbreak-rates-22-pre-deployment-training-data-filter.htm - 2026-10-30 [Industry; Press] Parse and Nomic are building a 100M-cell multimodal immune dataset with pharma partner commitments. (Charlie Roco). Deal: partnership . https://www.genengnews.com/topics/omics/100-million-cell-ai-ready-dataset-with-single-cell-and-proteomic-data-created-by-parse-and-nomic/ - 2026-10-30 [Industry; Press] Generative AI and prebuilt datasets drove Appen’s Q3 rebound, with new frontier lab clients but modest near-term revenue from them. (Appen). https://slator.com/appen-q3-2026-growth-accelerates/ - 2026-10-30 [Industry; Company] Abundant will design and audit RL environments for Fireworks customers’ workflow-specific post-training. (Abundant Team). Deal: working with . https://abundant.ai/blog/fireworks/ - 2026-10-30 [Industry; Company] Synthetic enterprise environments let Microsoft test Copilot workflows without using customer data. (Sophie Larsen). https://www.remio.ai/post/microsoft-copilot-synthetic-data-targets-enterprise-evaluations-privacy-gap - 2026-10-30 [Legal and Policy; Press] The DOJ probe centers on Nvidia controlling a key channel for AI models and datasets. (Tomislav Bezmalinović). Deal: regulation . https://www.heise.de/en/news/Hugging-Face-deal-DOJ-probes-Nvidia-s-expansion-strategy-11506261.html - 2026-10-30 [Legal and Policy; Press] A Hollywood-led amicus argues licensing creative works for AI training is feasible via existing collective licensing systems. (Human Consent Foundation). Deal: lawsuit . https://www.thewrap.com/industry-news/public-policy-legal/cate-blanchett-led-coalition-amicus-brief-new-york-times-openai-microsoft-battle/ - 2026-10-30 [Industry; Press] Bleu says abductive learning lets humanoids learn tasks from as few as ten expert demos, avoiding large task-specific datasets. (Bleu Robotics). Deal: funding €5 million. https://roboticsandautomationnews.com/2026/10/30/bleu-robotics-raises-e5-million-to-scale-humanoid-robot-deployment-in-factories/106554/ - 2026-10-30 [Release; X post] RL-trained LoRA adapters cut tool calls on the Pi harness without lowering accuracy. (Nathan). Result: Tool calls per task on Pi harness baseline -> 39% fewer than baseline (-39%), vs Base LFM2.5-24B at same task accuracy. Comparison: Vs. starting model. https://x.com/yb2698/status/2116114294383464836 - 2026-10-30 [Research; X post] NVIDIA shows agentic mid-training plus RL and on-policy distillation lifts SWE-bench Pro to 51.78% with fewer active parameters. (DAIR.AI). https://x.com/dair_ai/status/2116178819271032975 - 2026-10-30 [Research; X post] Short LoRA RL on Mistral Large 4 reportedly boosts AIME and aligns trainer-serving logprobs. (vLLM). Result: AIME 2024 avg@8 ~64% -> ~75% (), vs NVFP4 checkpoint before RL. Comparison: Vs. starting model. https://x.com/vllm_project/status/2115985388649242699 - 2026-10-30 [Release; X post] CATT shows many attribution methods miss known sources and proposes simple fixes. (Steven Yu). https://x.com/lamsheeper/status/2116183364394790938 - 2026-10-30 [Research; X post] Train a novelty-rewarded explorer and distill only verified traces into a student, then optimize without novelty to improve math reasoning. (alphaXiv). Result: pass@1 on five-benchmark mean 45.63% -> 51.81% (+6.18 pts), vs DAPO, matched 300 updates. https://x.com/askalphaxiv/status/2116078149612052645 - 2026-10-30 [Industry; X post] Factory bought Typedef to add a data context and lineage layer for enterprise agents and engineers. (Factory). Deal: acquisition . https://x.com/factory/status/2116214422779953574 - 2026-10-30 [Opinion; X post] Restricting LLM training data to pre-2022 aims to avoid LLM outputs in training. (Stella Biderman). https://x.com/BlancheMinerva/status/2116203742374613058 - 2026-10-30 [Research; X post] A simple Soft MTP tweak claims to give the KL self-distillation gradient for NTP without computing KL. (Jayden Teoh). https://x.com/jayden_teoh_/status/2116228700467302551 - 2026-10-30 [Release; X post] A new dataset offers 2 million multilingual safety prompts across 60 languages for alignment training. (Hanna Yukhymenko). https://x.com/a_yukh/status/2116214175064268860 - 2026-10-30 [Release; X post] An open-source bimanual UMI turns handheld demos into auto-uploaded LeRobot datasets, with real-robot success from VLA fine-tuning. (Amish Sethi). https://x.com/amish_sethi04/status/2116184130459205933 - 2026-10-30 [Release; Dataset card] A full public dump of agentic RL rollouts and SWE-Bench evals enables reproducible analysis of GRPO/GSPO training and verifier effects. (MasterZhou). Result: pass@1 on SWE-Bench Verified 25.70% -> 33.47% (+7.8 pts), vs base checkpoint, paired 498 instances. Comparison: Vs. starting model. https://huggingface.co/datasets/MasterZhou/agentic-rl-qwen3-30b-a3b-r2e-gym-rollouts - 2026-10-30 [Industry; X post] A vendor says it built 30+ RL environment types for frontier labs across coding, tools, browsing, and multimodal work. (Eigent AI). https://x.com/Eigent_AI/status/2116248711500099835 - 2026-10-30 [Research; X post] Self-verification with LLM-as-a-Verifier reportedly raised Nemotron 3.5 Super scores by 9 to 10% on agent benchmarks. (Jacky Kwok). Result: accuracy on Terminal-Bench 2.1 63% -> 72% (+9%), vs . https://x.com/jackyk02/status/2110866069137277081 - 2026-10-30 [Research; X post] Open-source task synthesis creates solvable, verified tasks that improve out-of-distribution agent performance. (Zhaoyang Wang). Result: average score across 8 OOD benchmarks -> (+10.58 pts), vs . Comparison: Vs. starting model. https://x.com/zhaoywang_CS/status/2112614057354125385 - 2026-10-29 [Opinion; X post] Every RL environment teaches the model habits, so good behavior has to be designed into the task, not only the reward. (Dimitris Papailiopoulos). https://x.com/DimitrisPapail/status/2115816854421639594 - 2026-10-29 [Research; X post] AI insiders see expert data vendors as the first place to sell a private dataset, according to an informal poll. (Chris Barber). https://x.com/chrisbarber/status/2115844044496752681 - 2026-10-29 [Release; X post] Games where time keeps moving reveal agent weaknesses that turn-based tests hide by pausing while the model thinks. (Sophia Xu). https://x.com/thesophiaxu/status/2115851937061183681 - 2026-10-29 [Opinion; X post] If training saturates an eval built to mirror real production work, the task is solved and the model should ship. (Florian Brand). https://x.com/xeophon/status/2115599055732764810 - 2026-10-29 [Industry; Company] Reddit's non-advertising revenue keeps growing, and earlier press coverage links that revenue line to selling its data for AI training. (Reddit, Inc. (SEC 8-K earnings release)). Deal: revenue milestone $48M other revenue in Q3 2026. https://www.sec.gov/Archives/edgar/data/0001713445/000171344526000114/earningspressreleaseq326.htm - 2026-10-29 [Legal and Policy; Press] Lawsuits over AI training data are speeding up, not settling down. (ChatGPT Is Eating the World). Deal: lawsuit . https://chatgptiseatingtheworld.com/2026/10/29/latest-map-of-u-s-copyright-suits-v-ai-companies-total-157/ - 2026-10-29 [Release; Dataset card] Long-form defensive-cyber SFT data with provenance and per-row metadata enables quality filtering and clean train/val/test splits. (Reizxn). https://huggingface.co/datasets/Reizxn/HolyCyber - 2026-10-29 [Research; Preprint] Multi-turn simulation RL with rapport-gated patients improves LLM psychiatric intake quality and safety in blinded psychiatrist evaluations. (Samuel Schmidgall et al.). Result: Differential diagnosis appropriate rate 33.3% -> 68.3% (+35.0 pts), vs Identically prompted base model. Comparison: Matched test. https://x.com/askalphaxiv/status/2116438362257326432 - 2026-10-29 [Research; Preprint] Dense LLM-judged rewards with online PPO on live iOS apps beat SFT and terminal-only RL for UI navigation. (Peitong Duan). https://arxiv.org/abs/2610.39264 - 2026-10-29 [Research; Preprint] Using generation-parameter distances for metric learning makes synthetic image pretraining transfer better than label-based classification. (Hina Otake and co-authors). Result: Recall@1 on Stanford Online Products 65.6% -> 74.0% (+8.4 pts), vs FDSL pre-training. Comparison: Matched test. https://arxiv.org/abs/2610.39423 - 2026-10-29 [Research; Preprint] Reshaping advantages by confidence alignment improves RLVR metrics while maintaining exploration. (Zhenghong Ke). Result: Pass@32 (Qwen3-4B-Base) -> (+2.4 pts), vs DAPO. https://arxiv.org/abs/2610.40407 - 2026-10-29 [Research; Preprint] Easy-start curricula often top in-distribution, but winners flip on held-out terrain and vary by robot. (Jakub Gilewicz). https://arxiv.org/abs/2610.40364 - 2026-10-28 [Release; Dataset card] Benchmarks should label whether an LLM judge or code scores each rubric item, and keep grading data hidden from the tested model. (HenryJ12 (individual Hugging Face user)). https://huggingface.co/datasets/HenryJ12/InvestorLastExam - 2026-10-28 [Opinion; X post] Keeping a test private stops labs from training on it, but it also loses the outside reviewers who find its mistakes. (Nathan Lambert). https://x.com/natolambert/status/2115437422146130032 - 2026-10-28 [Research; Company] SFT on graded teacher examples, inside the agent's real tools, produced most of one finance agent's improvement. (Samaya AI). Result: FrontierFinance score 42.5% -> 58.9% (+16.4 pts (SFT +12.3, RL +4.1)), vs base model; stage-by-stage, no matched RL-only control. Comparison: Vs. starting model. https://samaya.ai/blog/rl-post-trained-agentic-model - 2026-10-28 [Release; Dataset card] This dataset shows how changing a model's reasoning effort changes results on identical tasks, with full trajectories. (Kaiser Sun et al. (KaiserWhoLearns)). https://huggingface.co/datasets/KaiserWhoLearns/ReasiningEffortTrajec - 2026-10-28 [Research; Company] Expert rubrics catch models giving confident but outdated answers, especially on rules that changed recently. (Vals AI). https://www.vals.ai/blogs/can-ai-get-immigration-right - 2026-10-28 [Research; Company] Recognizing a dangerous chemistry request and refusing it are separate skills, and models often show only the first. (Madhu Sehwag). https://scale.com/blog/dualchem-evaluating-ai-models-dual-use-chemistry-risks - 2026-10-28 [Research; Company] In medicine, what ordinary people prefer is not a reliable sign that an answer is safe. (Anisha Osbourne). https://www.prolific.com/resources/what-s-new-edit-live-studies-run-frame-specific-video-evals-and-more - 2026-10-28 [Research; Company] Distillation works cleanly when only teacher answers passing every rubric check are kept. (Michael E., Ricky R., Nathan J.). Result: strict pass@1, held-out GDP.pdf 6.67% -> 11.83% (+5.16 pts), vs base Nemotron 3.5 Super VL. Comparison: No clear control. https://www.afterquery.com/blog/xtoken-distillation-gdp-pdf - 2026-10-28 [Industry; Company] Expert-built job simulations for training AI have become their own startup category. (Plato (press release)). Deal: funding $16M Series A. https://www.prnewswire.com/news-releases/plato-raises-16-million-to-build-the-training-grounds-for-ais-next-leap-into-real-work-302921832.html - 2026-10-28 [Industry; Press] Robot training data sellers are starting to compete on review quality and touch sensing, not just hours of video. (Sam Francis, Robotics & Automation News). Deal: funding $30M seed. https://roboticsandautomationnews.com/2026/10/28/maxinsights-raises-30-million-to-build-training-data-platform-for-physical-ai/106432 - 2026-10-28 [Legal and Policy; Press] How book rights are split between authors and publishers decides who can license books for AI training. (ChatGPT Is Eating the World). Deal: lawsuit $1.5B settlement fund. https://chatgptiseatingtheworld.com/2026/10/28/academic-textbook-authors-voice-concerns-about-their-publishers-seeking-85-cut-of-bartz-settlement-distribution/ - 2026-10-28 [Research; Preprint] Refreshing synthetic tasks based on reward dynamics boosts zero-data multimodal self-improvement without validation labels. (Yu Liu). Result: average across eleven benchmarks 57.77 -> 61.05 (+3.28 pts), vs before task refresh. https://arxiv.org/abs/2610.38568 - 2026-10-28 [Research; Preprint] A timestamp-frozen, expert-graded benchmark diagnoses where finance agents’ research fails to produce accurate forecasts. (Huangfei Jiang). https://arxiv.org/abs/2610.38342 - 2026-10-28 [Research; Preprint] Weakening a model to create an on-policy self-teacher boosts reasoning accuracy without privileged data. (Yecheng Wu, Song Han, Han Cai). Result: accuracy on HMMT 2025 (Qwen3.5-4B, pruning) 57.7% -> 70.5% (+12.8 pts), vs original model. Comparison: Vs. starting model. https://arxiv.org/abs/2610.38681 - 2026-10-27 [Research; Preprint] For tool-using AI, varying the tools and data in training environments matters more than adding tasks as budgets grow. (Andrew Szot, Alexander Toshev and coauthors). Result: AppWorld Test-N success (Qwen3.5-4B) 47.5% (zero-shot); 66.8% (SimSAT) -> 81.0% (SynthAPI-Big) (+14.2 pts over SimSAT), vs prior synthetic datasets' reported results (SimSAT, ToolACE, ToolAlpaca, Table 2); not matched on data size or teacher; SynthAPI-Big is the full dataset trained 4 epochs. Comparison: Cross-paper. https://arxiv.org/abs/2610.37648 - 2026-10-27 [Research; Preprint] If the preferred answer always comes from the same bigger model, training learns that model's style instead of real quality. (Giuseppe Curci, Andrea Zugarini and coauthors). https://arxiv.org/abs/2610.37058 - 2026-10-27 [Opinion; X post] When a model fails a benchmark question, the question itself is often broken, so check failures before trusting the score. (Hamel Husain). https://x.com/HamelHusain/status/2114935968604299552 - 2026-10-27 [Opinion; X post] Choose eval tasks that show which model is better, not just tasks that are hard. (Cameron R. Wolfe). https://x.com/cwolferesearch/status/2115121894328603063 - 2026-10-27 [Release; X post] Passing numerical tests does not prove that AI-written GPU code is actually correct. (Azalia Mirhoseini). https://x.com/Azaliamirh/status/2114886584290226217 - 2026-10-27 [Research; X post] Many workers cannot say what success looks like on their own tasks, or check it afterward, which makes AI training harder. (Mercor). https://x.com/mercor/status/2115122477567516803 - 2026-10-27 [Opinion; X post] Benchmarks that repeat one known model failure stop being useful the moment that failure is fixed. (Edward Hu (quoted by MTS)). https://x.com/MTSlive/status/2115223600391418263 - 2026-10-27 [Opinion; X post] Good evals should hide the real problem the way actual client requests do, then check whether the model finds it. (Edwin Chen). https://x.com/echen/status/2115189693159809472 - 2026-10-27 [Industry; Company] The headline AI model rankings from Artificial Analysis now rest mostly on tests that labs cannot see or train on. (Artificial Analysis). https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5 - 2026-10-27 [Research; Company] Judging whether code would actually be accepted needs project maintainers to write the rules that would block it. (Cognition Team). https://cognition.com/blog/frontier-code-2.0 - 2026-10-27 [Research; Peer-reviewed] Choosing proxy-run mixes so each data source's effect can be measured separately leads to better final training mixes. (Hubert Baniecki, Przemyslaw Biecek, Martin Pawelczyk). Result: MBPP pass@1 (4B Qwen3.5, 80B tokens) 0.304 -> 0.444 (+46.1% relative), vs Best Dirichlet-based baseline (RegMix, Olmix) and prior mix, same target model. Comparison: Matched test. https://arxiv.org/abs/2610.37390 - 2026-10-27 [Research; Preprint] Choosing a smarter data mix improves the final model only when the training data differs from what you want it to do. (Jiaxing Wang et al.). https://arxiv.org/abs/2610.36983 - 2026-10-27 [Research; Preprint] Replaying refusal examples keeps models safe but makes them refuse harmless requests, while distilling from the original model costs far less. (George Ruellan, Diana Benavides-Prado). Result: OR-Bench-hard over-refusal vs base (Mistral) +53.4 pts (replay) -> +5.1 pts (distillation) (48.3 pts less over-refusal), vs Supervised refusal replay at same task sequence. Comparison: Matched test. https://arxiv.org/abs/2610.38001 - 2026-10-27 [Release; Preprint] Some harmless-looking training examples weaken safety, and gradient attribution finds them when filters cannot. (Saket Reddy, Amro Abbas, Anshuman Suri, Ari Morcos, David Schwab, Pratyush Maini). Result: GCG attack success rate (Qwen) 47.7% -> 37% (-10.7 pts), vs Training on full Tulu 3; LLM-judge filter reached 47.3%. Comparison: Matched test. https://arxiv.org/abs/2610.37481 - 2026-10-27 [Research; Preprint] Plan information dependencies and repair tasks so professional agent tasks are executable, yielding better trajectories and stronger SFT models. (Ziyang Zeng). Result: Elo on GDPval-AA v2.1 879 -> 1565 (+686 Elo), vs base model. Comparison: Vs. starting model. https://arxiv.org/abs/2610.37161 - 2026-10-27 [Research; Preprint] Automated, source-grounded website reconstruction scales environments, beats WebHarbor fidelity, and yields training data that improves web agent success. (Tianci Xie). Result: WebArena-Verified Overall success -> (+13.33 percentage points), vs Qwen3.5-9B before fine-tuning. Comparison: Vs. starting model. https://arxiv.org/abs/2610.37786 - 2026-10-27 [Research; Preprint] A simulated SO-101 benchmark predicts real-robot rankings, enabling compute-matched evaluation of language-conditioned spatial manipulation policies. (Truman Hickok). https://arxiv.org/abs/2610.36420 - 2026-10-26 [Research; Company] An open model tried a rare unrequested upload in testing. Cohere cannot tell if it was planted or learned from agent logs. (Cohere (Cohere Team byline)). https://cohere.com/blog/concerning-behaviour-popular-open-weight-ai-model - 2026-10-26 [Release; Dataset card] NVIDIA's open coding agent trajectories now cover the same tasks run through five different agent harnesses, with reward-hacked runs removed. (NVIDIA (Wasi Uddin Ahmad, Nikolai Ludwig, Somshubra Majumdar, Boris Ginsburg)). https://huggingface.co/datasets/nvidia/Open-SWE-Traces - 2026-10-26 [Research; X post] Putting agents in separate virtual machines does not stop them from signaling each other if they share hardware. (Guillermo Rauch). https://x.com/rauchg/status/2114770919436808572 - 2026-10-26 [Release; X post] Reports of AI agents misbehaving often leave out basic facts, so a shared minimal template would help others learn from them. (Elie Bakouch). https://x.com/eliebakouch/status/2114768543954682282 - 2026-10-26 [Legal and Policy; Company] AI labs now tell investors that losing copyright cases could mean retraining models from scratch. (Anthropic PBC (Form S-1)). Deal: lawsuit . https://www.sec.gov/Archives/edgar/data/2130357/000162828026067168/anthropicpbc-sx1.htm - 2026-10-26 [Research; Preprint] A real-user multimodal coding benchmark with hidden criteria shows large gaps between criterion-level and full task success under agentic execution. (Xilin Wei et al.). https://arxiv.org/abs/2610.35138 - 2026-10-24 [Research; Preprint] Agent training data should cover tricky multi-step dependencies and keep only actions justified by what the agent could see. (Yihao Hu, Tao Lin and coauthors). https://arxiv.org/abs/2610.33568 - 2026-10-24 [Research; Preprint] Rewarding a model just for covering rubric items teaches it to pad answers, so precision must be rewarded too. (Hyun Ryu, Seungone Kim, Carolyn Rose, Eunho Yang and coauthors). Result: win rate vs base policy 10.6% RubricHub-Medical, 22.1% HealthBench (coverage-only reward) -> 68.0% (FORK), 74.1% on HealthBench (+57.4 pts; +52.0 pts), vs standard rubric-coverage reward, same policy model and grader. Comparison: Matched test. https://arxiv.org/abs/2610.33512 - 2026-10-24 [Industry; Company] Moonshot filtered cyber tasks out of its RL environments, not only out of its training text. (Kimi (Moonshot AI)). https://kimi.ai/blog/kimi-k3-1-pro - 2026-10-23 [Industry; Press] AI agents red-teaming copies of OpenAI's training setup escaped their sandboxes, so OpenAI now trains frontier models without internet. (OpenAI). https://openai.com/index/securing-our-frontier-model-training-environment - 2026-10-23 [Industry; Company] Baseten now refers customers to a few vetted RL environment builders, chosen mainly for process. (Charles O'Neill, Max Kirkby, Mudith Jayasekara, Ardin Lo). Deal: partnership . https://www.baseten.co/blog/introducing-the-baseten-environment-network/ - 2026-10-23 [Research; Company] A badly set up test can push an AI agent into harmful real-world actions, so tasks need checking before they run. (Paul C. Bogdan, Subhash Kantamneni and colleagues). https://www.anthropic.com/research/alignment-assessment-ukaisi - 2026-10-23 [Opinion; X post] Model builders tune to public tasks, and held-out splits drawn from the same public sets generally won't fix it, he says. (Brendan Foody). https://x.com/BrendanFoody/status/2113775443639960056 - 2026-10-23 [Industry; Company] Real business files, such as code and financial models, are becoming raw material for RL environments. (AfterQuery (Sid V., Arya F., Spencer M.)). Deal: data purchase referral payments up to $85k. https://www.afterquery.com/blog/announcing-partnerships-for-business-data - 2026-10-23 [Legal and Policy; Press] A first court ruling on whether training AI on song lyrics is fair use may now come. (ChatGPT Is Eating the World). Deal: lawsuit . https://chatgptiseatingtheworld.com/2026/10/23/judge-lee-concludes-marathon-hearing-on-summary-judgment-in-concord-music-v-anthropic-i-sets-2d-hearing-to-continue-daubert-hearing-on-oct-30-1-pm/ - 2026-10-23 [Research; Preprint] How heavily each data type's loss counts during training is a separate lever from how much of it you include. (Jiazheng Li, Huaqing Zhang, Jing Xu, Tianle Cai, Jingzhao Zhang). Result: Average over five benchmarks 21.43 (direct mixture, five-benchmark mean) -> 29.27 (LARE) (+7.84 pts), vs Matched direct mixture with same data exposure. Comparison: Matched test. https://arxiv.org/abs/2610.32141 - 2026-10-22 [Release; Company] SENSUS Bench scores whether an AI reply responds to how a person actually reasoned, not only whether it gives the right answer. (Lorm). https://huggingface.co/datasets/Lorm-Labs/sensus-bench-v1 - 2026-10-22 [Release; Dataset card] These rubrics were filtered only for valid formatting, not for meaning, so treat them as rough training signals. (NVIDIA (dataset contact Liwei Jiang)). https://huggingface.co/datasets/nvidia/Nemotron-System-User-Instruction-Hierarchy-v1 - 2026-10-22 [Release; Dataset card] To evaluate agents fairly in other languages, translate everything the agent reads, not just the opening prompt. (Qwen (Alibaba)). https://huggingface.co/datasets/Qwen/BabelArena - 2026-10-22 [Research; Company] Grading agents on the final database state and training only on medium-difficulty tasks improved pass@1. (Tuhin Kundu and Microsoft Copilot Studio team). https://huggingface.co/blog/microsoft/thinkingbox-training - 2026-10-22 [Research; Preprint] Choose an LLM judge by how good a model it helps train, not by its static benchmark score. (Angel Raychev, Nathan Lile, Rafael Rafailov). Result: peak policy Elo, Answer-Only SFT judge vs Thinking GRPO judge -> (Answer-Only higher in 7 of 8 matched settings), vs Thinking GRPO judges with higher static Gold accuracy; the comparison changes training objective and response format together. Comparison: Matched test. https://arxiv.org/abs/2610.30765 - 2026-10-22 [Research; Company] Safety monitoring must cover training runs too, and agents should not control when they get checked. (David Rein and Adam Kaufman). https://metr.org/blog/2026-10-22-red-teaming-openai-agent-monitoring/ - 2026-10-22 [Industry; Company] Hospital records are being packaged and sold as data for training and testing medical AI. (BioStack (press release)). Deal: funding $9.5M seed. https://www.prnewswire.com/news-releases/biostack-raises-9-5m-to-turn-global-hospital-data-into-multimodal-training-datasets-for-medical-ai-302916123.html - 2026-10-22 [Research; Preprint] On a GameCraft-Bench subset, schema-validated, frame-synchronous environments with adversarial reviews outperform other harnesses and distill reusable modules. (Wenbo Hu et al.). https://arxiv.org/abs/2610.33106 - 2026-10-21 [Opinion; X post] Human agreement is the anchor for judging open-ended work, and LLM judges are a calibrated stand-in, not ground truth. (Cameron R. Wolfe). https://x.com/cwolferesearch/status/2112915508148232571 - 2026-10-21 [Research; X post] If removing images does not change scores, a test is not really measuring image understanding. (sankalp (@dejavucoder)). https://x.com/dejavucoder/status/2112910513982292256 - 2026-10-21 [Opinion; X post] Seb Krier argues a recent AI agent incident came from several stacked training choices, each of which labs can control. (Seb Krier). https://x.com/sebkrier/status/2112931319835451671 - 2026-10-21 [Release; Company] To teach a chat model a new subject, keep general text in the mix and tune the learning rate first. (Hugging Face Bio (HuggingFaceBio)). Result: Average over five DNA benchmarks not reported (card states only the 27-point gap) -> 66.7% (+27 pts), vs Qwen3-14B starting checkpoint. Comparison: Vs. starting model. https://huggingface.co/HuggingFaceBio/Carbon-X - 2026-10-21 [Release; Preprint] Giving each prompt its own safety rubric, with listed exceptions, makes LLM-judge safety grading much more reliable. (Hongyu Li et al. (corresponding: Zhe Zhao)). Result: LLM-judge agreement with human annotation 69% to 87% (baselines) -> 95.34% (+8 to +26 pts), vs Baseline judging without prompt-level rubrics. Comparison: Matched test. https://arxiv.org/abs/2610.29225 - 2026-10-21 [Research; Preprint] Treat benchmarks as difficulty distributions and fix skew and gaps with targeted rewrites. (Yifan Sun). https://arxiv.org/abs/2610.29927 - 2026-10-20 [Research; Preprint] Small runs can predict when reward hacking will take over an RL run, before you train. (Junwei Deng, Alex Bie, Natalia Ponomareva). https://arxiv.org/abs/2610.28707 - 2026-10-20 [Release; X post] Let agents iterate against a validation set, then grade them on a held-out set they never see. (Justin Yang Chae). https://x.com/HuNtEr_J1324/status/2112544367650947554 - 2026-10-20 [Research; X post] On open-ended work, an agent's opinion of its own output can be far from what experts think. (Peter Kirgis). https://x.com/pkirgis/status/2112617994496991379 - 2026-10-20 [Research; X post] Judge AI agents on their full trajectories, not just the final result, and check that the monitors themselves work. (Arvind Narayanan). https://x.com/random_walker/status/2112509887456428208 - 2026-10-20 [Release; Dataset card] LLM judges of open-ended quality need checking against human labels before anyone relies on them. (Ai2 (Allen Institute for AI)). https://huggingface.co/datasets/allenai/novelty-judge-bench - 2026-10-20 [Industry; Company] Human data can be screened for AI use by watching how people behave, not only by reading their text. (Lily Haycraft Mee). https://www.prolific.com/resources/introducing-authenticity-checks-beta-ensure-genuine-human-responses-in-the-age-of-ai - 2026-10-20 [Industry; Company] Predicting how real people make choices is now measured with a benchmark built from real interview answers. (Rehearsals). Deal: partnership . https://fortune.com/press-releases/rehearsals-launches-human-behavior-bench-google-deepmind-2026-10-20/ - 2026-10-20 [Release; Company] Training AI agents now has an open, step-by-step recipe anyone can follow. (Ben Burtenshaw, Sergio Paniego, merve). https://huggingface.co/blog/huggingface/train-agents - 2026-10-20 [Research; Company] Hidden bias shows up when a model faces equal pressure for each candidate, not in its final picks alone. (Ai2). https://allenai.org/blog/fairstress - 2026-10-20 [Legal and Policy; Press] Courts are now tracing decisions about where AI training data came from back to named executives. (Diego Almada Lopez, Crypto Briefing (citing TorrentFreak)). Deal: ruling . https://cryptobriefing.com/meta-ordered-name-executives-ai-torrenting/ - 2026-10-20 [Legal and Policy; Press] Rights owners are using lost licensing income to argue that AI harms their market. (Mishcon de Reya). Deal: lawsuit . https://www.mishcon.com/generative-ai-intellectual-property-cases-and-policy-tracker - 2026-10-19 [Research; X post] A popular agent benchmark had flaws in most tasks, and once fixed, a top model nearly maxes it out. (Jennifer Zhou). https://x.com/chem_safety/status/2112212878132744674 - 2026-10-19 [Release; Company] Prime Intellect's RL framework now picks the same tasks for training and evals, and keeps timed-out rollouts out of training. (Prime Intellect Team). https://www.primeintellect.ai/blog/prime-rl-0-10-verifiers-0-4 - 2026-10-19 [Opinion; X post] Many company AI evals reject valid answers and pass wrong ones. (Brendan Foody). https://x.com/BrendanFoody/status/2112283673575948335 - 2026-10-19 [Research; X post] Capable coding agents perform better when given raw controls and left to build their own tools than when given simplified ones. (Sebastian Mueller (snimu)). https://x.com/omouamoua/status/2112248154813046938 - 2026-10-19 [Research; Company] How a model communicates, including sycophancy and slop, is now measured by LLM judges comparing answers in pairs. (Artificial Analysis). https://artificialanalysis.ai/articles/aa-personalitybench - 2026-10-19 [Research; Company] Scoring the process and the end result separately shows where an improvement really comes from. (Alex Gusak, Alex Petrov, Oleg Zubkov, Roman Smetanin). https://toloka.ai/blog/toloka-and-xmemory-test-what-memory-does-for-ai-agents/ - 2026-10-19 [Legal and Policy; Press] Buying books and scanning them for AI training now faces its own legal challenge, not just pirated copies. (Israr Ahmad, AllAboutLawyer). Deal: lawsuit . https://allaboutlawyer.com/meta-ai-copyright-lawsuit-hundreds-of-authors-llama-muse-2026/ - 2026-10-19 [Research; Preprint] Having an AI spot and penalize new reward hacks during training keeps training from falling apart. (Afonso Baldo, Antonio Farinhas, Marcos Treviso, Ricardo Rei, Nuno M. Guerreiro). Result: EQ-Bench Creative Writing v3 at final step 54.8 -> 60.5 (+5.7 pts), vs Same RL run with the reward model only. Comparison: Matched test. https://arxiv.org/abs/2610.26496 - 2026-10-19 [Research; Preprint] N-grams in a model's failed rollouts can pick a small, effective batch of extra pretraining text. (Xinyi Wang, Danqi Chen). Result: GSM8K accuracy after RL random pretraining text, same amount -> failure-retrieved text (+2.0 pts), vs Continued pretraining on equal random sample of pretraining corpus. Comparison: Matched test. https://arxiv.org/abs/2610.25836 - 2026-10-19 [Release; Preprint] Synthetic computer environments that pass strict quality checks can train agents that improve on tests they never saw. (Zhaoyang Wang, Qianhui Wu et al. (incl. Jianfeng Gao, Huaxiu Yao)). Result: Average over 8 OOD benchmarks (Qwen3.6-35B-A3B) 42.84 -> 53.42 (+10.58 pts), vs Same base model with the OpenCode harness. Comparison: Vs. starting model. https://arxiv.org/abs/2610.26828 - 2026-10-19 [Research; Preprint] Safety taught in midtraining holds up even when later tuning targets only general skills. (Zhengyue Zhao, Liwei Jiang, Yejin Choi, Chaowei Xiao). Result: Average attack success rate base model -> 72% to 81% lower (-72% to -81% relative), vs Qwen3 base models without safety mid-training. Comparison: Vs. starting model. https://arxiv.org/abs/2610.26809 - 2026-10-18 [Release; Dataset card] HackBench tests whether detectors can find reward hacking down to the exact command. (AI at Meta). https://huggingface.co/datasets/facebook/hackbench - 2026-10-18 [Research; Peer-reviewed] What counts as unsafe is a labeling choice, and safety benchmarks mostly measure which definition a model learned. (Vahid Jebraeeli, Jonathon Romero, Yijin Wang, Adrian Chan, Jianbang Zhang). Result: OR-Bench-hard over-refusal 3.4% (lowest arm) -> 56.4% (highest arm) (53.0 pts range), vs Ten arms varying how the two label policies are mixed, weighted or distilled; same architecture. Comparison: Matched test. https://arxiv.org/abs/2610.25060 - 2026-10-17 [Research; Preprint] A carefully tuned, plain RL setup can match methods that invent new training tasks, so tune the baseline first. (Boyi Liu, Yite Wang, Zhewei Yao, Yuxiong He et al.). Result: pass@1 on SWE-bench Verified (4B) 47.4% -> 65.0% (+17.6 pts), vs starting Qwen3.5-4B checkpoint, not a matched data or algorithm control; curated R2E-Gym, SWE-rebench V2 and Scale-SWE subset. Comparison: Vs. starting model. https://arxiv.org/abs/2610.24914 - 2026-10-16 [Release; X post] RL environments that read the real date can quietly become unsolvable as time passes. (Edgar Arakelyan). https://x.com/earakely/status/2111125700162031937 - 2026-10-15 [Research; Preprint] When LLM judges score rollouts against rubrics, the judge choice matters, and a confidence-weighted panel beats any single judge. (Michael Tian-Yue Liu, Anne Harrington, Trevor Darrell, Yutong Bai). https://arxiv.org/abs/2610.21179 - 2026-10-15 [Research; Preprint] Decontaminating math training data by deleting anything similar to benchmark questions throws away mostly good, unrelated items. (Esther Xin). https://arxiv.org/abs/2610.21229 - 2026-10-15 [Research; X post] Mercor says targeted expert data taught an AI agent to read source documents first and check the live system more often. (Mercor). https://x.com/mercor/status/2110756758272856439 - 2026-10-15 [Research; Company] Business tasks built backward from a known solution, with strict do-not-break rules, made effective training material. (Radhika Gaonkar, Florian Brand, Anirudh Ravichandran, Christian Munley, Daniele Molinari). Result: AutomationBench-AA 34.0% -> 57.7% (+23.7 pts), vs starting SFT checkpoint (step 0); single run, no matched control. Comparison: Vs. starting model. https://www.primeintellect.ai/case-study/nvidia - 2026-10-15 [Research; Preprint] Midtraining on real code history and agent logs makes coding agents better, even with identical SFT afterward. (Soumye Singhal, Joseph Jennings, Ka Shun Shum, Alexander Bukharin et al.). Result: SWE-bench Verified after identical full SFT 60.7 -> 64.5 (+3.8 pts), vs same base checkpoint without midtraining, identical full SFT; single run per arm, no seeds. Comparison: Matched test. https://arxiv.org/abs/2610.22019 - 2026-10-15 [Release; Dataset card] Finding requirements customers never state is the main hurdle for agents doing deployment work. (Dongxu Yang et al.). https://huggingface.co/datasets/DeepLethe/FDE-Gym - 2026-10-15 [Industry; Company] Customers get results by putting their own expertise into training data and rewards, not into prompts. (Thinking Machines Lab). https://thinkingmachines.ai/news/one-year-of-tinker/ - 2026-10-15 [Research; Company] When comparing a model's rollouts, full task completion should always count more than partial credit. (Mehdi Fatemi). https://snorkel.ai/blog/when-partial-credit-opposes-completion/ - 2026-10-15 [Research; Company] Training a model on its own successful moves, with hints removed, plus self-written lessons, beat training on curated teacher examples. (Romir Patel, Alex Dimakis, Maheswaran Sathiamoorthy et al.). Result: mean AutoResearchExam score, 22 CPU tasks 0.286 -> 0.358 (+24.8%), vs base Nemotron 3.5 Super VL. Comparison: No clear control. https://www.bespokelabs.ai/blog/teaching-nvidias-nemotron-ml-engineering - 2026-10-14 [Research; Preprint] Resettable copies of real apps can train phone agents, and the skills carry over to the real app. (MAI-UI Team, Alibaba Group (Panrong Tong and coauthors)). Result: task success on real train-ticketing app (physical phone) 28.8% -> 48.1% (+19.2 pts), vs same policy before clone-only RL (before/after, no separate control). Comparison: Vs. starting model. https://arxiv.org/abs/2610.20939 - 2026-10-14 [Release; Dataset card] NVIDIA released finance agent trajectories for SFT, keeping one best answer per question. (NVIDIA). https://huggingface.co/datasets/nvidia/Nemotron-SFT-Agentic-Finance-v1 - 2026-10-14 [Release; Dataset card] Creative 3D work can be graded by rendering it from several angles and checking many yes-or-no items. (Qwen (Alibaba)). https://huggingface.co/datasets/Qwen/BlenderWorld - 2026-10-14 [Research; Preprint] Small, randomized synthetic games teach control skills that carry over to real games the agent never trained on. (Mingyu Ouyang, Ying Shan, Mike Zheng Shou, Hwee Tou Ng et al.). Result: MineDojo-300 success (9B) 10.54% -> 38.33% (+27.8 pts), vs same model after SFT only (RL adds compute; not a matched-compute comparison). Comparison: Vs. starting model. https://arxiv.org/abs/2610.20584 - 2026-10-14 [Release; Dataset card] NVIDIA released a large set of AI-written tool-use conversations for SFT. (NVIDIA). https://huggingface.co/datasets/nvidia/Nemotron-SFT-Agentic-v4 - 2026-10-14 [Industry; Company] Simulated companies graded by automatic checks are now a packaged product sold by data vendors. (Labelbox). Deal: funding $100M+ Series E at $1.25B pre-money. https://labelbox.com/blog/labelbox-series-e - 2026-10-14 [Research; Preprint] Much of what audio models learn in post-training is benchmark-format patterns, not better listening. (Hua-Hsuan Chen et al. (incl. Hung-yi Lee)). Result: MMAU gain over base (Qwen2-Audio-7B, GRPO) +14.8 with real audio input -> +11.2 with silent input (76% of the gain remains without audio), vs Same post-trained model evaluated with silence instead of audio. Comparison: Matched test. https://arxiv.org/abs/2610.19476 - 2026-10-13 [Release; Dataset card] Building a hidden model of how a business works, then generating data and tasks from it, yields analysis tasks with checkable answers. (Zhanli Li and 18 co-authors). https://huggingface.co/datasets/Zhanli-Li/DataWorld-Bench - 2026-10-13 [Research; Preprint] Training agents with simulated users who reply and edit shared files improved them across six agent tests. (Anni Zou, Cheng Chen, Zhaoyang Liu, Bolin Ding, Jingren Zhou et al.). Result: mean score over six agentic benchmarks (Qwen3.5-9B) -> (+8.7 pts), vs base checkpoint; not a matched comparison with other environment data. Comparison: Vs. starting model. https://arxiv.org/abs/2610.17670 - 2026-10-13 [Research; Preprint] Start training with small rollout batches and grow them, because very large batches waste data. (Yuante Li, Junwei Quan, Kai Qiu et al.). https://arxiv.org/abs/2610.18853 - 2026-10-13 [Industry; Press] Workplace data from shut-down startups is becoming raw material for RL environments. (Matthew King, MIT Technology Review). Deal: data purchase about $25,000 for one startup code base. https://www.technologyreview.com/2026/10/13/1145755/ai-labs-buying-data-from-dead-startups-to-train-models/ - 2026-10-13 [Industry; Company] Open RL platforms are expanding into vision and computer use. (Prime Intellect Team). Deal: acquisition terms not disclosed. https://www.primeintellect.ai/blog/moondream - 2026-10-13 [Release; Preprint] Rubrics written from the instruction alone generalize better, and a judge with high benchmark accuracy may still train worse models. (Tianci Liu, Haoyu Wang). Result: Average downstream score after DPO 55.5 -> 57.9 (+2.4 pts), vs Rubric-ARM-8B pairwise rubric RM, same policy and DPO recipe. Comparison: Matched test. https://arxiv.org/abs/2610.18766 - 2026-10-13 [Release; Preprint] Accurate, varied GUI grounding data, including cases where the item is missing, beat larger open datasets. (Qingsong Xie et al. (incl. Ritankar Das)). Result: Average over four grounding benchmarks (SFT only) 73.7 (GroundCUA SFT) -> 76.4 (ScreenRef SFT) (+2.7 pts), vs Same model trained on GroundCUA under its recommended settings. Comparison: Matched test. https://arxiv.org/abs/2610.19100 - 2026-10-12 [Release; Dataset card] Public lab measurement databases can be turned into science questions with verifiable answers for RL. (NVIDIA). https://huggingface.co/datasets/nvidia/Nemotron-RL-litmus-bench-v0.2 - 2026-10-12 [Release; Company] Shared servers that handle sessions and verification let teams reuse their environment setups across many tasks. (NVIDIA NeMo team). https://github.com/NVIDIA-NeMo/Gym/releases/tag/v0.7.0 - 2026-10-12 [Industry; Company] Expert data vendors now have money to automate building RL environments, micro1's update suggests. (micro1). Deal: funding Series B at $4B valuation; $1B annual run rate. https://www.einpresswire.com/article/949082966/micro1-end-of-q3-update - 2026-10-12 [Industry; Press] Large-scale legal RL environments are now part of how AI labs build specialist products. (AI Weekly). https://aiweekly.co/alerts/zai-debuts-glm-legal-a-320b-legal-agent-built-on-glm-53-flash-with-192-harvey - 2026-10-11 [Research; Preprint] An automated loop where one model finds reward hacks and another patches the verifier makes verifiers much harder to game. (Chong Zeng, Aditi Raghunathan and coauthors). Result: verifier-aware per-trial attack success 82.4 / 99.3 / 83.3 (Sol / Opus / Gemini) -> 49.3 / 63.1 / 40.2 (33.0 to 43.1 pts lower), vs same tasks with original verifiers; benign per-trial solve within about 2 pts. Comparison: Vs. starting model. https://arxiv.org/abs/2610.15924 - 2026-10-10 [Research; Preprint] Building more simulated app worlds keeps improving AI assistant agents, so the number of worlds matters, not just tasks. (Mengzhou Wu, Yuzhe Guo, Linyi Li, Wei Yang, Tao Xie et al.). Result: average success on tau2-bench -> (+18.34 pts), vs same-size Qwen3 Thinking models without Terrarium RL; not a comparison with other synthesis pipelines. Comparison: Vs. starting model. https://arxiv.org/abs/2610.14860 - 2026-10-10 [Research; Peer-reviewed] Text, image and audio mixing ratios follow predictable scaling laws, and the three interact unevenly. (Aditi Khandelwal et al. (incl. Siva Reddy, Luke Zettlemoyer, Antoine Bosselut)). https://arxiv.org/abs/2610.14423 - 2026-10-09 [Research; Preprint] LLM judges agreeing with each other does not mean they agree with human experts. (Sangdo Han, Moonsoo Kim). https://arxiv.org/abs/2610.13303 - 2026-10-09 [Research; Preprint] Building RL environments works better as an agentic task, with feedback from the learner's rollouts, than as one prompt. (Simon Yu, Baolin Peng, Natasha Jaques, Weiyan Shi, Jianfeng Gao et al.). Result: Terminal-Bench 2.1 score (Qwen3.5-35B-A3B) -> (+4.3 pts), vs Tmax static pool regenerated with the same proposer model, matched training steps. Comparison: Matched test. https://arxiv.org/abs/2610.14163 - 2026-10-09 [Research; Company] Rewarding persistence in training can teach AI agents to get around restrictions in the real world. (Anthropic). https://www.anthropic.com/research/investigating-unintended-model-actions - 2026-10-09 [Industry; Press] Partnerships where robots are deployed for customers are becoming a source of training data for robot AI. (Jeff John Roberts, Fortune). Deal: partnership $62M raised; data terms undisclosed. https://fortune.com/2026/10/09/ultra-raises-62-million-fast-growing-robots-service-tie-up-ai-research-firm-physical-intelligence/ - 2026-10-09 [Release; Preprint] Copies of apps that can be reset and checked automatically make it possible to build large, verifiable training data for phone agents. (Chenyang Yan et al. (incl. Dahua Lin)). Result: SPA-Bench L3 real-device score (27B) 31.9 -> 59.6 (+27.7 pts), vs Qwen3.6-27B base model. Comparison: Vs. starting model. https://arxiv.org/abs/2610.14157 - 2026-10-08 [Research; Peer-reviewed] With a fixed budget for agent training data, new environments help most, then new tasks, then more rollouts per task. (Bowei He, Xue Liu and coauthors). Result: task success at identical total rollout tokens and gradient FLOPs -> (14 pts between two budget splits (Figure 1); recipe up to 1.7x compute efficiency, 4.0 to 8.4 pts at held-out scale), vs a different task/environment/trajectory split at same compute; naive scale-the-trajectories and Chinchilla-transferred allocations. Comparison: Matched test. https://arxiv.org/abs/2610.23946 - 2026-10-08 [Release; Dataset card] An environment should only include a task after its official solution passes the same verifier the agent will face. (FineEnvs (Adithya S K)). https://huggingface.co/datasets/FineEnvs/openai-math - 2026-10-08 [Research; Preprint] In RL, a problem teaches only when some rollouts succeed and some fail, so generate problems at that level. (Ximing Lu, Aviral Kumar, Jan Kautz, Yejin Choi, David Acuna). Result: mean accuracy over 8 math benchmarks 61.2 -> 69.8 (+8.6 pts), vs same gpt-oss-120b and 16K budget: full 78K corpus +1.1, best filtered subset +5.8; equal training compute per arm not stated. Comparison: Vs. starting model. https://arxiv.org/abs/2610.12659 - 2026-10-08 [Release; Dataset card] How hard a task is depends on the task, the model and the agent setup together. (ai& Research (Yagiz Calik, Jianbo Wu, Shimpei Hara)). Result: Terminal-Bench 2.1 pass@1 43.0 -> 46.1 (+3.1 pts), vs base model (Artificial Analysis figure); shipped checkpoint, mean of 7 evals; 5-run mean 45.1. Comparison: Cross-paper. https://huggingface.co/datasets/ai-and/termgrade-environments - 2026-10-08 [Release; Dataset card] Labeling reward hacking at each step of agent trajectories lets teams train and evaluate detectors for it. (Goodfire). https://huggingface.co/datasets/Goodfire/reward-hack-data - 2026-10-08 [Industry; Company] Data vendors now need research teams to prove that their data actually makes models better. (Mercor). Result: Pass@1 on APEX-Agents 16% -> 27% (+11 pts), vs Qwen3.5-397B before Mercor post-training. Comparison: No clear control. https://www.mercor.com/blog/why-mercor-is-building-a-research-team/ - 2026-10-08 [Research; Company] Unit tests no longer separate top coding agents, so engineering judgment must be scored with expert rubrics. (Surge AI). https://surgehq.ai/blog/sudo-l7 - 2026-10-08 [Research; Preprint] Have an agent try to reward hack your RL environments before the model being trained gets the chance. (Xiaomi LLM-Core Team). https://arxiv.org/abs/2610.11959 - 2026-10-08 [Industry; Company] How businesses grade their AI agents is starting to look like how RL environments grade models. (Labelbox). https://labelbox.com/blog/introducing-recursion-managed-agents - 2026-10-08 [Industry; Company] Ranking AI models by crowd votes has become a large paid business. (Arena Team). Deal: funding $200M Series B at $3.1B. https://arena.ai/blog/series-b - 2026-10-08 [Legal and Policy; Press] Using news articles for training without a license keeps drawing lawsuits seeking hundreds of millions of dollars. (Editor & Publisher (republished from Progressive Robot)). Deal: lawsuit over $250M sought. https://www.editorandpublisher.com/stories/usa-today-becomes-the-latest-publisher-to-sue-openai,263956 - 2026-10-08 [Release; Preprint] Training on many generated and tested synthetic apps teaches computer-use skills that work on real desktop software. (Tian Jin, Wenhao Wang et al. (corresponding: Siheng Chen)). Result: OSWorld Pass@3 41.39% -> 56.97% (+15.58 pts), vs Qwen3.5-9B base. Comparison: Vs. starting model. https://arxiv.org/abs/2610.12611 - 2026-10-07 [Research; Preprint] Fixed unit tests let coding agents pass while ignoring requirements, so fresh tests should be written for each attempt. (Shuangjie Yao, Baishakhi Ray, Dawn Song and coauthors). https://arxiv.org/abs/2610.10619 - 2026-10-07 [Research; X post] Most of Xiaomi's released coding environments still contain the correct fix, and its model found ways to use it. (Vals AI). https://x.com/ValsAI/status/2107967073729343806 - 2026-10-07 [Research; Preprint] On-policy distillation passes on reasoning skills but almost no facts. (Yixuan Tang, Yi Yang). Result: single-step fact recall (Qwen3-4B) 10.95% -> 54.10% (+43.2 pts), vs reverse KL vs forward KL, both on student prefixes, same teacher; controlled. Comparison: Matched test. https://arxiv.org/abs/2610.09639 - 2026-10-07 [Research; Preprint] Training agents against realistic simulated users, not overly helpful AI ones, helps them handle new kinds of users. (Hoang Phan, Dat Huynh, Yun He, Deren Lei et al.). Result: mean agent score across evaluation users 26.10 -> 29.54 (+3.44 pts), vs same GRPO trained against GPT-5.5 as user. Comparison: Matched test. https://arxiv.org/abs/2610.09484 - 2026-10-07 [Research; Company] Automated AI research still fails at rediscovering recent training methods. (David Owen). https://epoch.ai/publications/innovationeval - 2026-10-07 [Research; Company] Adding rubrics on top of a pass or fail check beats pass or fail alone, for equal training steps. (Simon Ivanov). Result: held-out OTS pass rate 41.6% -> 64.5% (+22.8 pts), vs base model. Comparison: Vs. starting model. https://toloka.ai/blog/what-one-epoch-of-our-ots-enterprise-rl-data-does-for-qwen3.5-27b/ - 2026-10-07 [Industry; Company] Robot training data from recorded human demonstrations now has its own large vendors. (Mecka (company announcement)). Deal: funding $60M Series B. https://www.mecka.ai/news/series-b - 2026-10-06 [Research; Peer-reviewed] Agent graders can pass runs that did the task but also changed things nobody asked for. (Richard Abrich). https://arxiv.org/abs/2610.09142 - 2026-10-06 [Research; Company] Tools for viewing AI agent logs can themselves be attacked, so agent output must be treated as untrusted. (David Rein). https://metr.org/blog/2026-10-06-ai-systems-could-cover-up-misbehavior/ - 2026-10-06 [Research; Preprint] Failed attempts can still teach a model if its after-the-fact reflections are turned into guidance. (Haoxiang Zhang, Silvio Savarese, Chien-Sheng Wu et al.). Result: success rate, 2B setting 0.0% -> 60.6% (+60.6 pts), vs RLVR alone, same rollout budget. Comparison: Matched test. https://arxiv.org/abs/2610.08077 - 2026-10-06 [Research; Preprint] A model given only a written behavior spec can create and grade its own safety training, and gains carry to agent tasks. (Jingyu Zhang, Shruti Palaskar, Daniel Khashabi, Benjamin Van Durme, Leon A. Gatys, Joseph Yitan Cheng). Result: Agentic Misalignment score 79.1 -> 3.8 (-75.3 pts), vs Qwen3.6-27B base. Comparison: Vs. starting model. https://arxiv.org/abs/2610.07935 - 2026-10-05 [Release; Company] Automatically generated RL environments now supply most training tasks for open AI models. (Reflection AI). https://reflection.ai/blog/introducing-beam - 2026-10-05 [Industry; Press] Consent and source tracking for data are being sold to AI companies as infrastructure. (Pulse 2.0 (relaying company release)). Deal: funding $400M commitment at $1B (cash share undisclosed). https://pulse2.com/signsplit-raises-400-million-strategic-seed-round-at-1-billion-valuation/ ## What Did Not Work - 2026-10-30 [Research; X post] In RL, a learned critic rarely beats spending the same compute on more rollouts. (Base Labs). https://x.com/baselabs/status/2116278798463492166 Why: Learned critics rarely beat spending the same compute on more GRPO rollouts - 2026-10-30 [Research; X post] When rival vendors publish test scores, rerun them yourself with the same settings before trusting the comparison. (Daniel Han). https://x.com/danielhanchen/status/2116162363313525191 Why: Unsloth could not reproduce rival quant scores, e.g. 1-bit AIME25 91.88% not 100% - 2026-10-27 [Research; Preprint] If the preferred answer always comes from the same bigger model, training learns that model's style instead of real quality. (Giuseppe Curci, Andrea Zugarini and coauthors). https://arxiv.org/abs/2610.37058 Why: Fixed-pair delta-learning DPO data underperforms multi-model preference data - 2026-10-15 [Research; Preprint] Decontaminating math training data by deleting anything similar to benchmark questions throws away mostly good, unrelated items. (Esther Xin). https://arxiv.org/abs/2610.21229 Why: Embedding-threshold decontamination removed mostly clean items; no threshold met 90% precision - 2026-10-09 [Research; Preprint] LLM judges agreeing with each other does not mean they agree with human experts. (Sangdo Han, Moonsoo Kim). https://arxiv.org/abs/2610.13303 Why: Rewriting rubrics raised judge-judge agreement but not agreement with human experts - 2026-09-30 [Research; Preprint] How rubric scores are aggregated matters, because simple sums let a model miss key points and still score well. (Maoqi Liu, Quan Fang and coauthors). Result: appropriateness on held-out physician criteria (Qwen3-4B) 54.1 untrained; 26.0 additive Rubric-RL -> 36.8 (ProRubric) (+10.8 pts over additive Rubric-RL, still below untrained 54.1), vs additive weighted-sum Rubric-RL with identical criteria, 3 seeds. Comparison: Matched test. https://arxiv.org/abs/2609.38847 Why: Rubric RL fell below the untrained model; ProRubric recovers part but stays below - 2026-09-26 [Research; Preprint] More game types help, but most of the gain comes from distillation, not the games. (Weixian Waylon Li, Fengxiang He, Yiwen Guo et al.). Result: 13-benchmark mean pass@1 (Qwen3-4B-Base) 60.8 -> 62.5 (+1.7 pts), vs non-game distillation control: same teacher, decoding, format and token budget; base model 45.5. Comparison: Matched test. https://arxiv.org/abs/2610.13995 Why: A non-game distillation control gets most of the gain; games add 1.7 pts - 2026-09-23 [Research; Company] Use an SFT warm-up on teacher examples only when the model cannot yet solve enough tasks to learn from rewards. (Base Labs (Charles O'Neill, Michael Psenka, Max Kirkby)). Result: post-RL strict accuracy, SFT+RL minus RL only -> (+11.3 (0.6B), +13.1 (1.7B), +2.1 (4B), -6.4 (8B)), vs GRPO from base checkpoint, matched at 80M RL response tokens; at matched update counts 4B also reverses (-5.69). Comparison: Matched test. https://labs.baseten.co/articles/when-does-distillation-help-reinforcement-learning Why: Distillation before RL hurt the 8B model by 6.4 pts at matched RL tokens - 2026-09-05 [Research; Preprint] Clever ways to pick or weight training problems did not beat simple random sampling in this study. (Hao Liang, Wentao Zhang and coauthors). Result: domain-balanced average accuracy -> (uniform GRPO +7.76 over base; selection/reweighting mean spread 0.97 pt and adaptive mixtures 0.61 pt, no paired CI excluding zero), vs uniform sampling and fixed equal mixture, same GRPO recipe. Comparison: Matched test. https://arxiv.org/abs/2609.06107 Why: No selection, reweighting or adaptive mixture beat uniform sampling or a fixed mixture - 2026-08-27 [Research; Preprint] Merging weights, mixed-data RL and multi-teacher distillation average about the same for combining specialists. (Siye Wu, Kai Yang, Yuchen Cai et al.). Result: multi-domain average -> (at most 1.4 pts between methods; up to 8.6 on one benchmark), vs Merge vs Mix RL vs MOPD, same experts and data (controlled comparison, no single baseline). Comparison: Matched test. https://arxiv.org/abs/2608.27409 Why: Merging, mixed RL and multi-teacher distillation tie within 1.4 pts on average - 2026-08-03 [Research; Preprint] Before adopting self-play, test whether the opponent itself actually helps. (Miseong Shawn Kim). Result: blinded head-to-head win rate, competitive vs non-competitive -> 49% (28:29, 9 ties; p about 1.0) (no reliable difference), vs otherwise identical non-competitive training run. Comparison: Matched test. https://arxiv.org/abs/2608.01559 Why: The self-play adversary gave no reliable benefit over identical non-competitive training