Review

Every example starts as pending, whether imported or generated. Only approved examples go into a version, and versions are what you train and evaluate on. Review is where you read each one, fix it and decide.

Open Review under Data. The queue is on the left, the example in the middle, and its source passage, evidence and quality checks beside it.

Before you start

• The project has examples, imported in Sources or generated with a recipe.
• You are a member or an owner. Viewers can read but not change examples.
• Optionally, a model endpoint to act as a quality judge.

Approve or reject an example

1. Choose an example in the queue. Filter by status, quality issue or category, or search with Search questions or answers (press /).
2. Read the conversation and the Source passage, and look at the quality checks.
3. Edit it if needed. See Edit an example.
4. Choose Approve or Reject, or press A or R.

Approve is unavailable while you have unsaved edits, while quality checks are running or have failed, or when a generated example fails its schema check or lacks a verified quote for an assistant turn.

You can change a decision at any time. Nothing in Review changes a version that is already frozen. When nothing is pending and something is approved, the header offers Freeze version.

Edit an example

• Conversation. Change the text of any message. You can't add, remove or reorder messages, and none can be blank. In a generated example that includes its source passage, leave the passage at the start of the first user message unchanged.
• Category. See Categories.
• Answer type for quality checks. Tells the checks what to expect: prose, a classification label, a number or JSON. Auto picks prose unless the answer is JSON or a number.
• Supporting source passages. See Source evidence.

Choose Save & recheck. The example goes back to pending and its checks run again. Changing only the category keeps its status.

Examples with images

Images in a vision model's examples show as thumbnails in the user message; choose one to see it full size. You can edit the text around them, but you can't change, move or remove images: to leave an image out, reject the example. Quality checks, search, redaction and similar prompts read only the text, so personal data inside an image is not detected, and a judge sees [image] in place of each image.

Source evidence

Source passage shows the text a generated example came from, with verified quotes highlighted. Supporting source passages lists the quotes for each assistant turn.

To add one, choose Add passage, pick the turn under Supports turn, paste the exact words into Exact source quote, and choose Save & recheck. A quote found word for word shows "Exact source passage".

Generated examples need a verified quote for every assistant turn. Imported examples have no passage, so their Source evidence score is 0%, but you can still approve them.

Quality checks

Scores run from 0% to 100%; higher means fewer issues found. They are signals for you, not approval decisions.

With a judge, three more checks appear: Factual support (every claim backed by the source, not just quoted), Answer completeness and Source consistency. Unsupported claims are listed under Flagged unsupported claims.

Recheck an example

Under Recheck with, choose Rule checks only or an endpoint as judge, then Recheck. The example goes back to pending. A judge's check runs in Activity, and the example can't be approved until it finishes.

What the judge receives

One request per example, containing the example's messages (with [image] for images), its evidence quotes and its source text. For a generated example, it also gets the project's goal and generation instructions and the recipe. The judge's provider sees all of this and bills you for each call. If the judge's answer isn't in the expected form, the check fails: Recheck, or use another judge.

Categories

A category names the kind of example, such as billing. Evaluations can break results down by category, and a version's coverage counts each category per split.

• Imported examples take their category column; generated ones take their source's category.
• For one example, type in Category and choose Save category.
• For several, select them, type in Category for selected (blank clears) and choose Set category.

Redact personal data

1. In PII and similar prompts, tick Possible email, Possible phone or both.
2. Choose Preview redaction to see each match replaced by [EMAIL] or [PHONE].
3. Choose Apply to draft, then Save & recheck.

This changes only the example. Source files and frozen versions are not changed. Names, addresses and other personal data are not detected.

Similar prompts

Exact duplicate prompts are always kept in the same split, but reworded near-duplicates are not. If one lands in training and the other in test, your test score looks better than it is.

1. Save any edits, then in PII and similar prompts choose Review similar prompts.
2. The console compares this example with 200 others at a time, oldest first, and lists those whose prompts share at least 70% of their words with this one, with their status and split.
3. Choose Next candidates to check the next 200.

The scan compares words, not meaning, and changes nothing. To keep one of two near-duplicates, reject the other.

Review several examples at once

1. Tick examples in the queue, or Select this page (X toggles the current one).
2. Choose Review selected, then Approve N or Reject N, or set their category.

If any selected example can't be approved, none are, and the console says why.

Keyboard shortcuts

Shortcuts are ignored while you type in a field.

Tips

• Read the answer, not just the quote. A verified quote shows where support should be, not that the answer is right.
• Keep answers consistent. The same question with different answers gets flagged under Possible contradictions. Settle on one answer, and fix or reject the other example.
• Remove near-duplicates before your first freeze with Review similar prompts, so test scores stay honest.
• Fill the answer type for labels and numbers, so very short answers aren't flagged by mistake.

Limits

When something goes wrong