• They train agents online on live iOS apps with a Planning agent and a UI Grounding model.
• An LLM judge supplies dense step-level feedback plus a terminal reward, optimized with PPO and step-level advantage estimation.
• Their 8B agents generalize to Android with nearly sevenfold gains over SFT and match Gemini-3.1-Pro on real iOS apps.

Setup: 8B · UINavBench, iOSWorld, AndroidWorld, and real iOS apps

• On ResNet-50, Recall@1 rose from 65.6% to 74.0% on Stanford Online Products over FDSL pre-training.
• On Stanford Cars, Recall@1 improved from 56.2% to 78.4% over FDSL pre-training.

Setup: ResNet-50 · Stanford Online Products and Stanford Cars image retrieval

• A-RSI uses reward variation and accuracy to decide task refreshes without validation or test scores.
• A static program library generates multimodal questions and reference answers, with audits and verified reward and refresh rule properties.
• Across eleven benchmarks, average performance rose from 57.77 to 61.05 after 800 fresh tasks.

Setup: primary eleven-benchmark suite

• ILE includes a versioned financial database with 3.43 million document versions across nine source categories, plus structured market data with validated timestamps.
• It organizes tasks by professional analyst deliverables and uses expert scoring linked to sources, calculations, and assumptions via financial dependency graphs.
• Evaluation covers 86 assignments and shows GPT-6 Astra and Claude Opus 5.5 score 68.1 and 66.9 overall, with Astra 52.5 on forecasting.

• WISD builds a weaker model via low-bit quantization or magnitude pruning, then uses the contrast as a training signal.
• Across math and coding benchmarks, quantization improves average accuracy by 4.0 points and pruning by 5.5 points over the original models.
• Pruning lifts Qwen3.5-4B from 57.7% to 70.5% on HMMT 2025 and from 39.1% to 56.7% on LiveCodeBench v6.

Setup: Qwen3.5-4B; Qwen3.6-35B-A3B · HMMT 2025; LiveCodeBench v6

• Apple's study covered 458,752 generated tasks and 849 trained models of four sizes.
• At larger data budgets, adding more kinds of tools and databases beat adding more tasks, and more complex tasks gave bigger gains.
• Agents trained following the resulting scaling law beat earlier synthetic-environment data on four benchmarks.

Setup: Qwen3.5 students (2B, 4B, 9B, 27B); Qwen3-4B/8B for BFCL comparison · AppWorld, BFCLv3 multi-turn, ASTRA-Bench, SynthAPI holdout

• Delta learning skips the judge and simply labels the larger model's answer as chosen in a fixed small and large pair.
• Models trained on it with DPO did worse than ones trained on multi-model data.
• Adding more answer-writing models partly fixed the problem, even without a judge.

• The usual method draws proxy mixes at random, which tangles the effects of different data sources together.
• LinMix includes each source either at its maximum share or not at all, so a simple formula can credit each one.
• On 1B and 4B models it beat random mixing by 46% relative on MBPP coding and 14% on GSM8K math.

Setup: OLMo2 1B; Qwen3.5 4B · HellaSwag, MBPP, GSM8K and other reasoning, code and math tasks

• With no train-target gap, tuned mixes tied uniform mixing on perplexity.
• They ended at 19.66 to 20.17 versus 19.71, and RegMix's 3.5-point lead midway vanished, so early gains may only mean faster training.
• The gap score, called Delta, is measured halfway through a run, and each experiment was run five times.

Setup: 160M and 1.1B LMs on SlimPajama-6B · Target perplexity

• On Mistral-7B, replaying refusal examples cut attack success on HarmBench by 51.1 points but raised over-refusal by 53.4 points.
• Distillation cut attack success 5.4 points and raised over-refusal only 5.1 points.
• Llama-3-8B showed the same pattern. Keeping the model's internal refusal direction did not reliably predict safe behavior.

Setup: Mistral-7B, Llama-3-8B (LoRA) · HarmBench ASR; OR-Bench-hard over-refusal; task scores

• DatologyAI removed the 20% of the Tulu 3 dataset that this method linked most to unsafe answers.
• An automated jailbreak attack's success on Qwen fell from 47.7% to 37%, while an LLM-judge filter only reached 47.3%.
• Removed examples included failed tool runs and arithmetic told as stories, and the models mostly kept their skills.

Setup: Qwen, Llama and Olmo models · GCG and ActorBreaker ASR; ORBench; MMLU Pro, IFEval, MBPP, GSM8K, MATH 500, MedQA

• They built 112,686 professional tasks across 612 occupations and collected teacher trajectories to fine-tune Qwen 4B, 27B and 35B-A3B.
• On GDPval-AA v2.1, WorkPilot-35B-A3B improved from 879 to 1565 Elo over the base model.
• Planning and repair reduced probe-diagnosed task environment defects and improved downstream performance.

Setup: Qwen-35B-A3B · GDPval-AA v2.1

• They automatically reconstructed 244 website artifacts across 14 domains and 53 categories, supporting 12,984 executed and accepted tasks.
• In a blind audit against WebHarbor, they achieved higher aggregate fidelity on three metrics, with 12 wins, two ties, and no losses.
• Fine-tuning Qwen3.5-9B on 5,856 trajectories raised REAL success by 9.9 percentage points and WebArena-Verified overall success by 13.33 percentage points.

Setup: Qwen3.5-9B · REAL, Odysseys-Easy, WebArena-Verified

• The benchmark includes 916 evaluation episodes, 138 objects, and 3,629 simulated demonstrations for fine-tuning under a matched compute budget.
• Policy rankings transfer to the physical arm with Spearman 1.00 and MMRV 0.000, though simulation overestimates success rates.
• Across nine learned models, best policies reach about 61% success, while an LLM agent with a grasping policy scores 83.2%.

Setup: SO-101 Bench; success in sim and transfer to real robot

• It samples 273 tasks from 37,463 deduplicated queries across 26,314 real user coding sessions.
• Tasks include runnable pre-edit apps and 2,331 hidden expert criteria across multimodal editing and repository editing tracks.
• Under a shared agent harness, the best of 10 endpoints averages 69.9% criteria but fully solves 26.4% of tasks.

Setup: Co-Design Bench

• Trajectory2Gym builds runnable environments aimed at multi-step patterns that existing training data rarely covers.
• It trains only on decisions supported by the history visible to the agent at that moment.
• Average scores on the Tau1, Tau2 and BFCL tool-use benchmarks rose by 16.9 to 29.8 points across five Qwen3 model sizes.

• Models trained with RL for rubric coverage added padded or wrong content and lost to the original model.
• Adding a precision reward, called FORK, raised the win rate against the original model from 10.6% to 68.0% on RubricHub-Medical.
• On HealthBench the win rate rose from 22.1% to 74.1%, with similar rubric coverage and shorter, more precise answers.

Setup: Qwen2.5-7B-Instruct policy; Qwen3.6-35B-A3B training grader · RubricHub-Medical test set, HealthBench; win rate vs base policy

• LARE keeps each data type's share fixed, but adjusts how heavily its loss counts.
• It sets those weights automatically from running loss values, so no extra search runs are needed.
• On a 1.5B model trained on 240B tokens, it beat a matched mix by 7.84 points on average.

Setup: 1.5B GPT-2-style decoder · MMLU, GSM8K, MATH, HumanEval, MBPP

• Answer-only judges trained better writing models than more accurate thinking judges in 7 of 8 matched settings.
• Bigger judges kept helping, even when they were larger than the model being trained.
• Results rest on one learned 32B model acting as final referee, and a follow-up check was inconclusive.

Setup: Qwen3 judges 0.6B to 14B; Qwen3 policies 0.6B to 8B · creative writing policy Elo under a learned 32B Gold evaluator

• The engine stores project state as schema-validated structured text and offers frame-synchronous runtime control for inspection, action execution, and behavior verification.
• An adversarial team of eight specialized agents separates implementation from review and fixes problems through multi-round loops.
• On a GameCraft-Bench subset, VibeGame in P2G outperforms other agent harnesses, including GPT-6.1-Sol, using the same underlying model.

Setup: GameCraft-Bench subset

• Each of 3,489 English and Chinese prompts gets its own rubric that scores safety and usefulness separately.
• Listed exceptions stop judges from wrongly flagging acceptable answers as unsafe.
• Rubric-guided LLM judges agreed with human labels 95.34% of the time, versus 69% to 87% for earlier methods.

Setup: Judge agreement with human labels; model unsafe rates

• They formalize two failure modes: measure mismatch and coverage gaps, with two diagnostic metrics and theoretical guarantees.
• They analyze 25 benchmarks across 7 domains using item difficulties, finding skewed difficulty weights and uncovered portions of the difficulty axis.
• BenchFill rewrites items to target specific difficulties and reduces local coverage gaps.

Setup: 25 popular benchmarks across 7 domains

• SimHack predicted the median start of reward hacking within 11.3% relative error on training recipes it had not seen.
• Its ranking of recipes had a 0.85 rank correlation with actual results, while the starting hacking rate alone did not predict onset.
• Curricula chosen with SimHack trained 96.6% longer before reward hacking took over.

• Training only against the reward model raised its score while real eval scores fell.
• Versus reward-model-only training, live penalties raised final scores from 3.37 to 3.80 on clinical counseling and 54.8 to 60.5 on creative writing.
• A reward model retrained with about 15,000 extra comparisons was about as accurate but still got gamed in new ways.

Setup: Qwen3.5-35B-A3B · MindEval; EQ-Bench Creative Writing v3

• Failed rollouts during RL share telltale n-grams, from reflection phrases to specific operations.
• Text containing them, just 35 million tokens, beat the same amount of random text in 10 of 11 settings.
• It added 2.0 points on GSM8K math when used before midtraining. Text picked for OLMo2-1B improved Qwen3-4B by 1.9.

Setup: OLMo2 and Qwen3 models · GSM8K; math, commonsense and knowledge mixture; math and science

• The 341 worlds mix GUIs, MCP tool servers and command lines, with long multi-step tasks.
• Five checks run each task to confirm it is solvable, hard enough, and graded consistently by different judges.
• Qwen3.6-35B's average on 8 unseen agent benchmarks rose from 42.84 to 53.42, and removing any check cut scores.

Setup: Qwen3.6-35B-A3B, Qwen3.5-4B · Online-Mind2Web, WebArena, QwenClawBench, ClawEval, CUA-Gym, OSWorld-Verified, OSWorld 2.0, Agents' Last Exam

• The base model wrote its own safety reasoning documents for 185,000 prompts, about 200 million tokens after an LLM judge filtered them.
• Attack success rates fell by 72% to 81% versus Qwen3 base models from 1.7B to 8B, while general usefulness improved.
• Safety held through later SFT and RL on general skills only.

Setup: Qwen3-1.7B, 4B, 8B; Llama3.2-1B, 3B base · ASR on malicious requests; over-refusal; utility suite

• Lenovo labeled the same 53,000 or so prompts under two definitions of unsafe and trained ten guardrail model versions.
• Over-refusal ranged from 3.4% to 56.4%, while recall on harmful requests stayed the same.
• The training data already contained 88% of one over-refusal benchmark and 82% of another.

Setup: Ten guardrail arms at two model scales · HarmBench, WildGuard, OR-Bench-hard, JailbreakBench

• Snowflake tuned its setup by deliberately overfitting a small task pool, which exposed failures that a rising reward score hid.
• A 4B model reached 62.0% on SWE-bench Verified, near a rival's 61.5% with one sixth the rollouts.
• The rival comparison comes from a different paper, not a matched run, and curating three public task sets lifted scores further.

Setup: Qwen3.5-4B and Qwen3.5-9B · SWE-bench Verified, SWE-bench Pro v1 and v2

• Training Qwen3-4B on medicine and science rubrics, simply switching among five judges produced a large spread in scores.
• A plain average of the judges did not beat the best single judge.
• RaRE weights judges by confidence and skips answers they disagree on, beating every single judge on HealthBench and GPQA Diamond.

• At a similarity cutoff of 0.80, 52 of 59 sampled removals, or 88%, were confirmed to be wrongly removed.
• No similarity cutoff between 0.80 and 0.98 reached the target of 90% correct removals.
• Having an LLM judge review likely matches reached about 98% precision and caught about 98% of real matches.

• After identical SFT, Lightning rose from 60.7 to 64.5 on SWE-bench Verified, improving on every coding benchmark.
• In a smaller test, removing the agent work logs erased most of the gain, scoring 60.5 versus 63.5 with them.
• An RL check, one 20-step run, kept the advantage, but gains from that stage are unproven.

Setup: Nemotron 3.5 Lightning and Nemotron 3.5 Super · SWE-bench Verified, Multilingual, Pro; DeepSWE

• Training in a mock train-ticket app raised success on a physical phone from 28.8% to 48.1%.
• Alibaba's MAI-Gym runs up to 20,000 copies at once and verifies task success by execution.
• Its 9B model scores 78.4% on MobileWorld and 76.2% on OSWorld-Verified, and 2,000 verified tasks are released.

Setup: MAI-Gym-9B from Qwen3.5-9B (SFT warm-up then RL) · MobileWorld, OSWorld-Verified; 52 paired tasks in real 12306 app on a physical phone

• Player-1 used SFT on 6,044 gameplay recordings, then RL in 16 synthetic games.
• The synthetic games target control skills common across games and randomize what the agent sees and how things move.
• RL added 27.8 points on MineDojo for the 9B model and 17.7 for the 35B model.

Setup: Player-1-9B and Player-1-35B-A3B · MineDojo-300 success

• Qwen2-Audio-7B gained 14.8 points on MMAU after training, and still kept 11.2 points when the audio was replaced by silence.
• Models trained entirely on silent clips still improved on real audio tests.
• Most gains disappeared when multiple-choice questions were changed to open-ended ones.

Setup: Qwen2-Audio-7B-Instruct, Qwen2.5-Omni-3B and open models · MMAU, MMAR, open-ended variants

• Alibaba built 64.9K work environments across 14 scenarios, repairing broken ones in a build, run and revise loop.
• Average gains over the starting model were 8.7, 6.4 and 4.1 points for the 9B, 27B and 35B-A3B sizes.
• The 27B model ranks second on the six-test average, 0.6119 versus 0.6157 for GLM-5.2.

Setup: Qwen3.5-9B, Qwen3.6-27B, Qwen3.6-35B-A3B · Workspace-Bench, ALE, WildClawBench, PawBench, ClawEval, ClawMark

• Rollout batches of 1,536 rollouts did as well as 512, but larger batches needed more total samples for the same progress.
• Training collapsed after roughly the same number of samples no matter the batch size, across a 16x range.
• Starting small and switching to large before predicted collapse matched the best fixed batch size.

Setup: Qwen2.5-Math-1.5B; checks on Qwen2.5-1.5B-Instruct and Qwen2.5-Math-7B · math RLVR accuracy

• ReVeR writes rules from the instruction before seeing answers, then scores each answer on its own.
• Its small 4B grader trained better models than larger 8B graders that compare answer pairs, averaging 57.9 versus 55.5.
• Training the rubric generator with RL raised the judge's benchmark accuracy but made it follow rubrics less faithfully.

Setup: Qwen3.5-4B RM; Qwen2.5-7B-Instruct policy · RewardBench, RM-Bench, RewardBench 2, RubricBench, JudgeBench; IFEval, IFBench, MulDimIF, Arena-Hard

• ScreenRef pairs 586,000 screenshots with over 633,000 descriptions of on-screen items, including cases where the item is absent.
• The same model scored 76.4 on average across four benchmarks after training on ScreenRef, versus 72.9 and 73.7 for two rival datasets.
• The authors think the missing-item examples likely explain their edge on OSWorld-G, one of the benchmarks, but did not test this directly.

Setup: Qwen3-VL-8B-Instruct · ScreenSpot-Pro, OSWorld-G, MMBench-GUI L2, ScreenSpot-V2

• On 102 Terminal Bench tasks, reward-hacking success by stronger, unseen models fell by 33.0 to 43.1 points.
• Honest solve rates stayed within about two points of the original.
• Without validation checks, honest solve rates were 74.8% by the second round, versus 92.8% with the full method.

Setup: GPT-5.6-Luna hardener (Terra for KernelBench); GPT-5.6-Sol, Claude Opus 5, Gemini 3.1 Pro attackers · 102 Terminal Bench tasks; also KernelBench and SETA

• Terrarium built 275 resettable app worlds and 13,344 tasks, graded on the end state rather than a reference trajectory.
• RL on them raised average scores 18.34 points on tau2-bench and 10.94 on AppWorld over untrained Qwen3 models.
• For Qwen3-4B, scores kept rising as worlds grew from 35 to 275, but comparisons are to untrained models, not other methods.

Setup: Qwen3 4B, 8B, 14B · tau2-bench, AppWorld

• Researchers ran 268 training runs from 457M to 8.3B parameters on up to 150B tokens.
• Repeating image or audio data more than twice added little. Image data hurt audio, but audio barely affected text or images.
• The scaling law predicted eight of nine results for three new test runs within 2.6%.

Setup: MoE models 457M to 8.3B (validation up to 12.6B-A3.8B) · Per-modality validation loss

• Researchers rewrote the rubric parts a five-judge LLM panel disagreed on most, without using human ratings.
• On the SummEval summarization benchmark, agreement among judges rose, but agreement with expert ratings did not change.
• Moving some criteria into fixed code checks also brought no improvement.

• An agent working in a terminal produced up to 25 times more working environments, at up to 75% lower cost each.
• Training on them beat fixed task pools on Terminal-Bench 2.1 by 4.3 to 7.4 points, depending on model.
• Without feedback from the learner's rollouts most of the gain disappeared, and Terminal-Bench 4.0 rose only 1.6 to 3.2 points.

Setup: Qwen3.5-9B, Qwen3.5-35B-A3B, Qwen3.6-35B-A3B · Terminal-Bench 2.1 and 4.0

• It includes 35 copies of commercial-style apps, over 110 apps in total, and over 300 app tools across more than 50 apps.
• It releases nearly 12,000 SFT sessions and over 2,000 RL tasks with automatic verification.
• The resulting agent scored 79.9 on AndroidWorld, and real-device SPA-Bench scores rose from 31.9 to 59.6.

Setup: Qwen3.5-9B and Qwen3.6-27B (OpenMobile-2-9B, -27B) · AndroidWorld, MobileWorld, SPA-Bench, MobileGym++ Bench

• This order held across all model sizes, families, domains and both RL methods tested.
• The environment advantage comes mainly from varied worlds. Varying environment structure helps about as much as new tasks.
• A budget plan built on these findings was up to 1.7x more compute efficient than just adding rollouts.

Setup: Qwen3-8B default; Qwen3-4B/14B, Llama-3.1-8B, Qwen2.5-7B · held-out task success across math+code, SWE, web, UI/tool (AppWorld, ALFWorld) and search

• About 88% of a large pooled math dataset was solved by every rollout or by none, giving the model nothing to learn.
• Growing new problems from hard seed problems raised gpt-oss-120b from 61.2 to 69.8 across eight math tests.
• Training on the full 78,000-problem dataset gave only 1.1 points, and the best filtered subset gave 5.8.

Setup: gpt-oss-120b, DeepSeek v4 Flash, Qwen3-4B-Instruct · 8 math benchmarks (AIME24/25, AMC, HMMT, Minerva, Olympiad, Omega); SecQue, FinanceBench

• Environments were stripped of logs, patches and later git history, and tests had to pass eight reruns.
• Cheats found before training get zero reward if used later, and confirmed cheats stayed below 2% of rollouts.
• The training mix was 68% coding, 13% design, 12% tool use, 4% cyber and 3% following context.

• SoftwareGym built 182 working apps in 12 categories, repairing each until its screen tasks passed, with a 91.5% average pass rate.
• Training on these apps raised Qwen3.5-9B's OSWorld pass@3 from 41.39% to 56.97%.
• No OSWorld apps or tasks were used in training, and a 120-task benchmark of its own was also released.

Setup: Qwen3.5-9B · OSWorld Pass@3 (eight app domains); SoftwareGym-Bench

• About 34.4% of attempts judged correct actually broke the task requirements.
• The overall success rate fell from 50.6% to 33.2% once these were caught.
• The study covered 6 frontier models and 5 benchmarks, including DeepSWE and SWE Marathon.

• With the usual reverse KL objective, Qwen3-4B recalled 10.95% of facts, versus 54.10% with forward KL.
• Reverse KL still raised skill at combining steps on new problems from 1.8% to 76.8%.
• The same split appeared on real factual questions and competition math.

Setup: Qwen3-0.6B, Qwen3-4B, Gemma-2-2B, OLMo3-7B · synthetic fact and composition tests; Knowledge-2026 factual QA (809 items); Math-2026 (201 problems)

• On tau-bench, a GPT-5.5 agent succeeded 63.6% of the time with real users but 82.4% to 84.4% with AI-played users.
• With the same training method, swapping GPT-5.5 for MIMESIS-9B as the user raised the average score from 26.10 to 29.54.
• These agents beat GPT-5.5-trained agents under all nine unseen simulated users, by 1.46 to 4.55 points.

Setup: MIMESIS-9B simulator; Qwen3-8B agent trained with multi-turn GRPO · eight Gym environments (three held out), nine unseen evaluation user models

• WorkArena's form grader passed forms with an unrequested extra field in all 36 test pairs.
• In two AppWorld task templates, tests passed a duplicate write in every task.
• One added count check per template rejected all six duplicates and passed all six correct runs.

• GRPO learns by comparing attempts, so when every attempt in a group gets the same reward, it learns nothing.
• Between 37% and 98% of groups showed no reward difference.
• For a 2B model with 98% of groups all failing, plain RL reached 0.0%, while adding the method reached 60.6%.

• The model writes its own ethical dilemma tasks and rubrics, then trains on them with itself as the LLM judge.
• Qwen3.6-27B's misbehavior score on the Agentic Misalignment test fell from 79.1 to 3.8.
• Training used only single-turn chat, yet improved behavior in agent tasks, and AgentHarm harm fell from 22.6 to 14.8.

Setup: Qwen3.6-27B · AgentHarm; Agentic Misalignment; general capability

• A teacher that sees the rubric trains the student on its own answers through on-policy distillation, before rubric-based RL.
• On HealthBench, Qwen2.5-7B reached 0.673 versus 0.607 with an SFT warm-up, and Llama-3.1-8B reached 0.634 versus 0.526.
• SFT warm-ups taught models to claim rubric compliance without content, and a score of 0.83 fell to 0.40 when regraded.

Setup: Qwen2.5-3B, Qwen2.5-7B (Qwen2.5-32B teacher), Llama-3.1-8B (Llama-3.1-70B teacher) · HealthBench, ResearchQA, RubricHub Science, GPT-4o-mini shortcut-aware grading

• Arena trained a reward model on about 5.6 million human votes between image pairs.
• Using rubrics as gates with a veto worked better than simply averaging them with the vote score.
• Win rate over the base rose from 0.537 with 10K pairs to 0.607 at 100K, 0.611 at 500K and 0.642 at 5.6M.

Setup: FLUX.2-dev and Ideogram-4 · Arena text-to-image leaderboard; MMRBv2 pairwise win rate vs base

• Models trained on real human preferences predicted choices better than rubric sets and code-based verifiers.
• Gaps appeared even in math and software, and were largest for peer review and creative writing.
• Picking outputs by written criteria drifted away from what people actually preferred.

Setup: Gemma-family judge; Llama-3.1-8B, Qwen2.5-14B, phi-4 generators · Prediction of human preference labels across 42 tasks

• CUE compresses a person's chat history into an embedding, then turns it into a short persona guide.
• It works with any LLM user simulator, was fit mainly on customer support, and was tested in other areas.
• On tau2-Bench it caused fewer simulator errors than persona-based methods, and the model and examples are released.

• DeskForge varies windows, overlap and resolution, and labels items using the system's accessibility data and window positions.
• Training on 200,000 examples raised Qwen3.5-4B's ScreenSpot-Pro score from 30.17 to 41.68.
• With a fixed planner, the model completed 50 of 119 WebArena-Infinity tasks, up from 31.

Setup: Qwen3.5-4B and three other VLMs · ScreenSpot-Pro, ScreenSpot-v2, OSWorld-G, UI-Vision, MMBench-GUI; WebArena-Infinity, OpenApps

• In medical consultations, summed rubric rewards raised coverage, while appropriateness on separate physician criteria fell below the untrained model.
• Regrouping the same items recovered about a third of the loss, while shorter answers recovered almost none.
• ProRubric, which counts groups of items only when all hold, raised appropriateness by 10.8 points without losing coverage.

Setup: Qwen3-4B and Qwen3-8B (non-thinking) · appropriateness on held-out physician criteria (medical common set, DeepSeek-V4-Pro judge) and rubric coverage; seven-benchmark average

• Rejected answers from a small Qwen2.5-3B model improved a 14B student by 3.1 points on code and 1.9 on math.
• Raising the share of rejects from a 1B model from 0% to 100% lifted code scores from 59.73% to 63.28%.
• The pattern held for students from 7B to 72B, and less likely answers made more useful contrasts.

Setup: Qwen2.5-Instruct students 7B to 72B; Qwen2.5 and Llama reject sources · code generation and mathematical reasoning, avg@4

• The pairs that look most informative may mostly reflect where the judges are biased.
• The NAOD method estimates the reward and each judge's bias together.
• On Chatbot Arena data, it cut average regret by 29.1% versus a matched method.

Setup: Proxy policy on Chatbot Arena archive · Proxy policy regret; held-out human preference prediction

• With the agent fixed, changing the simulated user shifted average task scores by 15.2 points across 375 tasks.
• 24.4% of successful runs included the simulated user breaking its instructions.
• The most common slip was revealing information too early, which cut the agent's tool use by 1.06 calls on average.

• Gains grow with the square root of the smaller of log(comparisons) and the KL budget.
• In a synthetic setup, a 70B reward model labeled the data, and smaller reward models from 0.6B to 4B learned from it.
• The scaling law fit with R2 of 97% to 99%.

Setup: Proxy RMs 0.6B to 4B; 70B gold RM · Gold reward after best-of-N or policy tilting

• Adding more compute to a single midtraining run stops helping after a point.
• Branches started from the same checkpoint with different recipes end up usefully different from each other.
• Merging each branch's few best checkpoints beat the best single run at the same token budget, and kept improving.

Setup: Ling-3.0-Tiny (7.9B total, 1.3B active MoE) by default · Aggregate downstream performance

• The authors checked 3,810 passing attempts from 29 model and test pairs, separating real solves from unearned passes.
• The most common route was finding reference solutions in the code's git history.
• On SWE-Bench Pro, unadjusted confirmed violation rates rose from 24% to 73% between two models, then fell to 11% and 0%.

• Seven experts spent about a year producing about 2.8K example answers and 5.4K preference pairs in Arabic and English.
• In blind expert review, SFT on example answers beat the baseline 51.3% to 14.4%, a statistically clear result.
• Adding DPO on top won 28.0% to 20.9%, which was not statistically significant (p = .166).

Setup: QCRI post-training pipeline (base model not named in abstract) · Blind expert pairwise evaluation on 150 held-out prompts

• Varied examples leave the model succeeding and failing on more prompts, which gives later training more to learn from.
• OLMo3-7B's pass@8 rose 16.9 points on unseen environments.
• A cheap rule-based selector that runs on CPUs beat more expensive selection methods on three open datasets.

Setup: OLMo3-7B; Qwen3 students · RLVE environments held out from SFT (pass@8); 10 math benchmarks (mean pass@8)

• An AI wrote 336 checked game variants, and distillation from a teacher playing them lifted a 4B model from 45.5 to 62.5.
• Distillation from the same teacher without any games reached 60.8, about 15.3 of the 17-point gain.
• Self-play RL added another 2.4 points, reaching 64.9.

Setup: Qwen3-4B-Base, Qwen3-8B-Base, Mistral-Nemo-Instruct-2407, GLM-4-9B-0414; Qwen3-32B teacher · 13 benchmarks, pass@1 at 8,192 tokens

• Using the full HH-RLHF prompt set hurt the model's coding and instruction following.
• HelpSteer3 prompts were hard for the reward model to judge.
• The final stage used 75,815 filtered HH-RLHF prompts out of 112,052 originals.

Setup: GLM-4.5-Air-Base (106B-A12B) · Stagewise benchmarks vs official GLM-4.5-Air release

• Taste-Bench tests the judgment calls agents make inside long, multi-step tasks.
• The best model, GPT-5.6 Sol, answers 59.7% of its questions correctly.
• With a trained student's advice, an agent reached 33.7% on unseen SWE-bench Pro tasks, 19.1 points above no advice.

• onPanda lets reviewers fix a model's own outputs token by token, for both chat models and agents.
• In a small controlled study, median review time fell 52% compared with editing whole answers by hand.
• The team released a dataset called Panda-CVL and a test for token-level correction, but the abstract gives no dataset size.

• One hundred expert players recorded 5,000 hours across 21 AAA games.
• An offline test has thousands of questions across several time spans, with three main tasks plus diagnostic versions.
• The team tested 47 models with over one million calls. A revised version was posted on 2026-10-12.

• Agents read existing code, describe what features do, and write tests by running the original code.
• Tasks were filtered by pass rate over repeated rollouts, giving 5,545 tasks from 3,185 repositories in 23 languages.
• Training on them improved MiMo-V2.5 on DeepSWE by 11.7% and ProgramBench by 17%.

• Agents get a running app on Ubuntu, macOS, Windows, Android or Web and must build a faithful copy.
• Hidden behavior tests come from the original app and were checked by human reviewers.
• GPT-6 Astra leads RecreationBench's 250 tasks at 58.1%, but passes every test on only 2.8% of tasks.

Setup: Frontier and Qwen models · RecreationBench; ProgramBench, GameCraft-Bench, Vision2Web, OSWorld 2.0, WeaveBench

• Models trained both ways scored the same at first, and differences appeared only after the same later training.
• It costs no extra data, model size or text.
• At 4B, Terminal-Bench 2.0 pass@1 rose 29% relative. At 8B, pass@1 dipped but pass@16 rose.

Setup: Qwen3-4B and Qwen3-8B · Terminal-Bench 2.0; aider-polyglot (225 tasks)

• Researchers tested models up to 110 billion parameters with up to 1 billion tokens of alignment documents.
• The documents steered the model's motives only in simple, ambiguous situations.
• A tiny share of fine-tuning data suggesting a different motive erased the effect, and rules stuck only when actually demonstrated.

Setup: Models up to 110B parameters · Motivation and rule-following probes

• ImpossibleRubrics has 169 impossible tasks, where the only honest answer is saying it cannot be done, plus 48 answerable controls.
• Rubrics from eleven AI generators rewarded misleading answers 8 to 26% of the time.
• On a stress test, the strongest generator was fooled 36% of the time, versus 0% for a proof-faithful rubric.

• Partial teacher reasoning raised pass@8 from 2.2% to 85.4% as hints grew longer.
• On Qwen3-1.7B across nine benchmarks, the average was 36.8 at base, 44.3 with GRPO and 47.3 with fading hints.
• SFT on the full teacher reasoning reached 39.8, 7.5 points behind the fading-hint method.

Setup: Qwen3-1.7B and Qwen3-0.6B · 9 benchmarks incl. MATH-500, AMC23, AIME24/25/26, GPQA-D

• None of eight selection or weighting methods clearly beat uniform sampling, and none of three adaptive mixes beat a fixed equal mix.
• Plain uniform training improved balanced accuracy by 7.76 points over the base model.
• Method rankings flipped with the test suite, correlating at -0.33 between math-heavy and balanced summaries.

Setup: Qwen2.5-7B-Base (Llama-3.1-8B-base extension) · 12 math, logic and science benchmarks, domain-balanced average

• Researchers split safety responses into the stock refusal sentence and the explanation of why.
• Stock refusals made models react to surface words like 'shoot', while explanation-only training cut false refusals at similar safety.
• Llama-3.1-8B's compliance with safe requests rose from 0.39 to 0.71, while harmful compliance went from 0.02 to 0.06.

Setup: Llama-3.1-8B, Mistral-7B-v0.3, Gemma-2-9B, Qwen2.5-7B base models · XSTest-Safe and OKTest compliance; AdvBench, HarmBench, JailbreakBench, WildJailbreak, SORRY-Bench

• On Qwen3-8B, refusal-only data raised target refusals from 9.47% to 84.75%, but XSTest over-refusal from 2% to 74%.
• Using the model's own verified responses cut over-refusal from 15.20% to 5.20%.
• Paired harmful and harmless examples cut held-out over-refusal from 32.94% to 4.16%. Harmful refusals dipped from 91.88% to 87.72%.

Setup: Qwen3-8B · Target-domain refusal; XSTest; three harmfulness benchmarks

• Distillation from a post-trained teacher's token probabilities kept improving reasoning but slowed factual recall.
• Teachers were less confident on knowledge-heavy text.
• Their Switch Distillation method reached 1.61 to 1.71 times the reasoning of normal training while keeping 96.7 to 96.8% of factual recall.

Setup: OLMo-2 1B student · Reasoning, knowledge and commonsense, factual recall

• Across seven leading agents, 57.1% of runs used the shortcut.
• Five of the seven agents took it in more than half of their runs.
• Even when told not to, the average cheating rate stayed above 50%.

• A 4B teacher's feedback pointed the wrong way 30.6% of the time, yet removing that noise did not change results.
• Simply penalizing low-probability tokens by a fixed amount matched the teacher's feedback.
• A teacher-free version lifted Qwen3-1.7B's AIME24 math score to 48.85, versus 32.08 with a teacher, and longer answers did not explain it.

Setup: Qwen3-1.7B student; Qwen3-4B-Instruct teacher for OPD · AIME24, AIME25, HMMT25 (Avg@32 and Pass@32)

• During training, each answer pair is treated as correct, flipped, or a tie, based on how strongly the model prefers one answer.
• Across 57 settings, the method won 60.5% against standard DPO, versus 55.5% for the next best.
• The baseline is one round of DPO on UltraFeedback Binarized, and the study also analyzes human disagreement.

Setup: Multiple models (Table 1) · Win rate vs DPO on UltraFeedback, AlpacaEval, AlpacaEval 2, others

• Using the same specialist models and data, average scores differed by at most 1.4 points across methods.
• Gaps on single tests reached 8.6 points, tracking how the specialists' weight changes relate to each other.
• None of the methods measurably widened the range of problems the model could solve, or lost skills on unseen tests.

Setup: Qwen3-4B-Instruct-2507 and Qwen3-8B (non-thinking) · multi-domain benchmark suite; experts for five domains

• Comparing the best and worst of several answers showed that weak prompts cap answer quality.
• Researchers picked weak prompts by reward score and rewrote them with rubric-guided AI feedback, without dropping examples.
• Gains were clear in offline training but mixed in online training, and some measures fell when a reward model supplied labels.

Setup: Llama3-8B, Mistral-7B · MT-Bench, Evol-Instruct, AlpacaEval 2.0

• SkillCorpus collects 38,040 public multi-file skill packages, which describe how to use tools without needing environments.
• Public skill packages on npm grew from 170,226 in February 2026 to 1,640,440 in June 2026.
• Training on them ranked first in 23 of 24 agent comparisons and beat direct fine-tuning by 9.11 to 24.96 points.

Setup: Three model scales · Four agentic benchmarks plus general benchmarks

• The authors argue RL mostly shapes how a model chooses among skills it already has.
• About 30% of a large environment pool, with weak ones removed and graders fixed, gave 106% larger gains on Terminal-Bench-Lite.
• Results averaged three runs across 2B to 27B models, but rival numbers were copied from its report, not rerun.

Setup: Qwen3.5 2B, 4B, 9B and Qwen3.6 27B (TMax comparison); Qwen3-8B main recipe · Terminal-Bench-Lite, Terminal-Bench v2.1

• The 20.3B-token mix is 42% web, 23% PDF, 26% code and 9% agent logs, with synthesized tool workflows.
• With the same fine-tuning, Qwen3-4B's BFCLv3 tool-use score rose from 39.73% to 50.25%, and to 54.18% with RL.
• Midtraining on the same amount of general Dolmino data reached only 43.10%.

Setup: Qwen3-4B-Base, Qwen3-8B-Base · BFCLv3, tau2-Bench, MCP-Universe

• One AI model both writes runnable RL environments with graders and learns in them through RL.
• The designer's reward is the gap between the learner's scores with and without hints.
• At 30B size it beat the best fixed pool by 5.3 points on eight unseen tests. Document grounding and memory were critical.

Setup: Qwen3-4B-Instruct-2507, Qwen3-8B, Qwen3-30B-A3B-Instruct-2507 (primary) · eight held-out math, science, code and reasoning benchmarks; BFCL v4; ACEBench-Agent

• For each starting task, the measured pass rate decides whether to rewrite it harder, easier or different.
• With the same budget of 100 environments, it beat the best fixed recipe by about 2 points on command-line tests.
• Qwen 3.5 35B rose from 40.0% to 49.2% on one test, and its SWE-bench Verified score reached 77.1% versus 73.4%.

Setup: Qwen 3.5, 4B to 35B · Terminal-Bench core and 2.0; SWE-bench Verified

• The author compared training against an opponent with an otherwise identical run that had no opponent.
• Four separate tests all found no reliable benefit. A blinded head-to-head ended 28 wins to 29, with 9 ties.
• An early 29% advantage turned out to be small-sample noise, and a pilot with a stronger opponent tied 32 to 32.

Setup: legal argument quality, blinded pairwise judgment

• Qwen-CUA sees only screenshots and uses keyboard and mouse, with no access to page code or accessibility data.
• It trained on about 40,000 checkable tasks using nearly 100,000 cloud CPUs, refreshing data and retuning tasks between rounds.
• It scored 86.2 on OSWorld-Verified and cut red-team attack success from 36.6 to 16.4 versus Qwen3.7.

Setup: Qwen MoE 397B-A17B · OSWorld-Verified, OSWorld 2.0, RedTeamCUA and five others