Push model accuracy past imitation with verifiable rewards.
Go beyond supervised examples when correctness is checkable. The model samples multiple candidate answers, a built-in metric or sandboxed Python function scores each from 0 to 1, and GRPO reinforces what wins.
Why reinforcement learning raises the ceiling
Imitation learning plateaus when gold answers are hard to author or multiple valid paths exist. Reward-driven training optimizes the actual grader.
Direct alignment
Optimize the metric you actually care about
Align model weights directly toward valid JSON schemas, exact numeric tolerances, classification accuracy, or custom Python checks.
Safe extensibility
Bring custom Python logic without infra work
Write a clean score(answer, reference) function; Tensorant runs it inside an isolated WASI/sandbox environment with zero credentials exposed.
Zero regression
Automatically keep the highest-scoring checkpoint
Every round evaluates on a fixed validation sample so late-run drift or time limits never overwrite your strongest adapter.
What RL is good for
RL fits answers a rule can check: a number, a label from a list, JSON that follows a schema, or anything a short Python function can score. The reference can be just the part the reward checks.
Train toward what you measure
By default the reward is your baseline's scoring and main metric, so training pushes toward what you measure. Change it to another built-in metric, such as Token F1, classification accuracy, JSON-schema correctness or numeric accuracy.
Read the docsWrite the reward in Python
Register a score(answer, reference) function that returns a number from 0 to 1. It runs in a sandbox on the GPU machine, in processes that hold none of your credentials, and the run keeps the version you saved.
Read the docsStart from your SFT adapter
RL learns from the differences between answers, so it works best when the model already gets some right. Train with SFT first for the shape of the answer, then continue that adapter with RL.
Read the docsTry one round first
Set the rounds, prompts per round and answers per prompt, then run a trial: one round on up to 8 prompts. It reports peak GPU memory, the seconds each part took, and an estimate of the full run's time and GPU cost.
Read the docsOnly the best round graduates to production.
Each round scores the same sample of up to 128 validation prompts, and the run keeps the adapter of the best round. Even if a run reaches its time limit, it stops early cleanly and preserves its peak checkpoint while holding out your test split.
- Tensorant compute or your GPUs
- Train on Tensorant compute with credits, or in a GPU account your organization connected with the Own compute subscription.
- On every organization
- RL is available on every organization with clear preview estimates before launch.
- A portable adapter
- Test, deploy, benchmark with Inference Autoresearch, publish, or download like any LoRA adapter.
Next step