Scoring and ranking
Score and rank candidates with a calibrated decision model
Fine-tune a ModernBERT or Ettin encoder such as kev to score each candidate answer with a calibrated probability. For routing, triage and moderation.
The problem
Why not let a chat model pick?
A chat model answers with text it writes. Asked to pick a route, it can name one that is not on your list, and it gives your options no probabilities that your code can compare with a threshold.
For routing, triage, moderation and scoring, you already know the options. You need a probability you can act on, such as sending a case to a person when the top answer is below 70%. See When to use a decision model.
How it works
What a decision model does
- A decision model reads a text, such as a support ticket, with a typed question and its candidate answers. It scores every candidate and returns a probability for each, calibrated with a temperature fitted on your data.
- It never answers outside its candidates. The candidates can differ from one question to the next, because the model reads them with the question. See When to use a decision model.
- A run trains on one GPU, with LoRA or full training. It scores the base model and the tuned model on your test cases, and fits the temperature on the validation split. See How a run works.
Models
Models to start with
Tensorant trains decision models on ModernBERT and Ettin encoders of up to 1.2 billion parameters. See Supported models.
skundu42/kev
A ModernBERT scorer of about 400 million parameters, fine-tuned from the Ettin encoder below. Training continues from its one-score head and its own temperature.
jhu-clsp/ettin-encoder-400m
An Ettin encoder. Training adds a new head that gives each candidate one score.
answerdotai/ModernBERT-base
ModernBERT's base model. Training adds a new one-score head, as it does for the Ettin encoder.
Steps
How to do it in Tensorant
Create a decision project
Enter a decision model as the Hugging Face base model when you create the project. A project's kind comes from its base model and never changes.
Import decisions or labelled texts
Import typed decisions, each a state with its questions and the gold answer to each, or labelled texts, such as a CSV with
textandlabelcolumns. See Decision projects.Freeze a version
Approve the cases and create a version. It keeps each case whole, and cases that share a state in one split, so no test case shares a state with a training case.
Train and evaluate
Configure a decision run. It scores the base model, trains, fits the temperature and scores the tuned model, with accuracy, log loss, Brier score and calibration error for each.
Serve it or download it
Publish the model under a name and call it on
POST /v1/decisions. See Serve decision models. Or download it as a ZIP that kev loads, or publish it to your Hugging Face account. See The result.
Plans
What you need
- Decision models are on every organization: creating the project, launching its runs and serving its models. A model you trained downloads, or publishes to Hugging Face. See Images, RL and decision models in Pricing and billing.
- Each training run, and each run that only scores a model, uses one GPU. A training run pays the training-token fee for the candidate pairs it trains on; a run that only scores a model pays none. See Before you start.
Fine-tuning
Train a calibrated decision model with kev
Fine-tune kev, a 400M ModernBERT scorer, on typed decisions, check calibration quality, and gate the run with measured scores and cost.
Deployment
Serve a decision model and call it from your app
Deploy a trained decision model, call POST /v1/decisions with batches and retries, and act on calibrated probabilities with measured cost.
Private and sensitive data
Medical, legal and financial records: what Tensorant stores, where it runs, and what can stay in your cloud.
Classification and routing
Answer with one label from your list: intents, ticket routes, what needs a person.
JSON extraction
Pull the fields your code needs out of invoices, forms and emails, as JSON that fits your schema.
Replace a costly LLM API
Train an open model on the answers your API model gives, and switch when it scores as well.
Small language models
Fine-tune a 1B to 4B model for one narrow job, and download the adapter.
Reinforcement learning
Train with GRPO against a reward that checks each answer: a label, a number, JSON or your own code.
Migrate from OpenAI fine-tuning
OpenAI ends fine-tuning in January 2027. Move your data, train an open model and compare the two.
Start with one dataset.
Create an account and import your data, or book a demo and we will show you the product.