RL environments

Rewards you can check

The hard part of reinforcement learning for agents isn’t the algorithm — it’s the environment. If the reward can be gamed, the policy will game it. So the work here is building environments where success is verifiable: a task either got done or it didn’t, and the signal comes from ground truth rather than a model’s opinion.

The approach

Start from tasks with a clear pass/fail: a test that runs, a value that matches, a state the world is or isn’t in. Wrap them so an agent can act, observe, and be scored automatically — then let the policy improve against a reward it can’t fake. The emphasis is on environments that mirror how an agent is actually used: reaching for tools, taking several steps, and being judged on the outcome.

Environments

Tool use

Agents that call real tools and APIs, rewarded on whether the end state is correct — not on how plausible the transcript looks.

Code

Write it, run it, grade it against tests. The reward is a test suite that either passes or doesn't — as objective as signal gets.

Retrieval

Find the right thing in a large corpus and use it. Success is checkable against ground truth, so reward hacking has nowhere to hide.

Multi-step planning

Long-horizon tasks with intermediate checkpoints, where the reward tracks whether each step genuinely moved the goal forward.

Why it runs on our own compute

RL is a tight loop — the model acts, the environment scores it, the policy updates, repeat, endlessly. That rewards having accelerators you own and can saturate rather than rent by the minute. It’s the reason the Compute thread exists, and where these environments will train as the hardware comes online.