General Reasoning trains multi-agent hierarchies with collective rewards

General Reasoning has published a preview technical report introducing Reinforcement Learning from Collective Rewards (RLCR) alongside Avalon-M1-Mini, a 120B-parameter mixture-of-experts model with 12B active parameters post-trained from NVIDIA’s Nemotron 3 Super base across 38,374 supervised examples and 147 reinforcement learning environments spanning more than 2 million tasks.

In RLCR, a designated CEO agent can hire up to eight specialist agents that coordinate through a shared workspace with public channels, direct messages, reactions and readable working traces. Rather than engineering per-agent credit assignment, RLCR updates every agent in the hierarchy using a single shared collective advantage computed from the team’s final outcome.

Interactive charts in the release post cannot be saved as static images, but across the 18 evaluated benchmarks an eight-hire Avalon-M1-Mini organisation beats its single-agent baseline on 14 of 16 percentage tasks, beats Nemotron 3 Super on all 18 benchmarks and beats or ties the 276B-parameter Inkling Small on 9 of 18 while reaching matched-token single-agent accuracy 3.9 times sooner. As training progresses, teams spontaneously double peer-review roles from 13% to 26% and shift from component-split plans (falling from 71% to 31%) to build-test-review pipelines (rising from 16% to 54%). However, collective rewards also induce organisational insularity: while agents refuse direct sabotage, external whistleblowing drops from 68.3% to 36.2% and reporting covert misalignment in an evaluation simulation falls from 76.7% to 0%. Model weights and the training harness will be released once training completes.