Recipes

A recipe tells a model what kind of examples to write from your documents: factual questions, troubleshooting steps, JSON extraction and more. When you generate, each selected document is cut into passages and each passage is sent to a model endpoint with the recipe. The examples come back to Review as pending, each with exact quotes from its passage as evidence.

You only need recipes for documents. Imported question and answer pairs are already examples.

Before you start

• You are a member or an owner.
• The project has documents in Sources.
• Your organization has a model endpoint that speaks the OpenAI chat completions format, or a ready deployment.
• You are happy for that endpoint's provider to see your documents. See What is sent to the endpoint.

Starter recipes

The project's use case decides which starter the generation form offers first. You can generate with a starter directly, or save your own recipe from one.

Save a recipe

1. Open Recipes under Data, and choose a starter under Starter recipes (or New recipe for Factual Q&A).
2. Enter a Recipe name.
3. Write the Generation instructions in your own words. Left empty, the starter's instructions are used.
4. Choose the Training context. See below.
5. For Structured JSON, enter the Response JSON schema. See Structured JSON.
6. Optionally tick up to five Approved references. See Reference examples.
7. Choose Save recipe.

To change a saved recipe, choose it, edit it and choose Save new revision. Jobs and versions keep the revision they used, and each example shows its recipe and revision in Review. Saved recipes cannot be deleted.

Training context

• Include source passage (recommended). The passage is put at the start of each example's first user message, followed by the question. The model learns to answer from text it is given. Choose this if your application will send relevant text along with each question.
• Closed book · omit source passage. Examples hold only the question and answer, so the model learns to answer from memory.

With the source passage, leave the passage in the example unchanged. Editing it makes the example fail its checks.

Reference examples

References show the endpoint the style you want: tone, length, format. Tick up to five approved examples under Approved references. They are copied into the recipe and sent with every passage, and the endpoint is told to copy their style, not their facts.

• References must be approved training examples of this project, 20,000 characters together, without images.
• A reference, and anything sharing its source or question, always stays in the training split of every version, so you are never tested on an example the generator was shown. See Split reservations.

Structured JSON

A Structured JSON recipe needs a Response JSON schema (draft 2020-12, inline; no $ref, pattern or format). The starter asks for one text field:

{
  "type": "object",
  "properties": {"answer": {"type": "string"}},
  "required": ["answer"],
  "additionalProperties": false
}

Replace it with the fields you extract. In Review every answer must match the schema before it can be approved.

Generate examples

1. In Sources, tick whole imports or single sources.
2. Choose Generate examples.
3. Choose the Recipe: a starter or a saved recipe.
4. Choose the Generation endpoint.
5. Optionally choose an Optional quality judge. It adds one model call per example to check factual support, completeness and contradictions. Rule checks only adds no call.
6. Set Examples per passage (1 to 10, default 3).
7. Choose Generate for N.

Follow the job in Activity.

How generation runs

• Each source is cut into 6,000-character passages, and one request goes to the endpoint per passage.
• New examples arrive in Review as pending, with their source's category. Exact duplicates are not added.
• Every assistant turn needs an exact quote from the passage. A generated example can't be approved until each turn has one. A quote shows where support should be, not that the answer is right, so read the answer too, or add a judge.
• A failed passage is retried a few times. Examples from earlier passages stay, and Retry in Activity continues from the failed passage.

What is sent to the endpoint

For each passage, the endpoint receives the passage (up to 6,000 characters), the project's Goal and Generation instructions, the recipe with its full reference examples, and fixed instructions to treat all of it as data and quote the passage exactly. With a judge, there is one more request per example; see What the judge receives.

Tips

• Start small. Generate from a few sources, read the results in Review, adjust the instructions or references, then generate for the rest.
• Use a capable generator. Each answer must be a JSON list of examples with exact quotes. If passages keep failing, switch to a more capable model.
• Mix recipes. A few Insufficient information examples alongside factual ones teach the model to say when the source doesn't cover a question.

Limits

When something goes wrong