Automation and configuration

Use JSON or YAML configuration files for repeated operations, and --json for results consumed by a script. Keep configuration in version control; supply credentials through profiles or secret environment variables.

Configuration files

Training and evaluation commands accept a flat object matching their MCP tool's arguments. Unknown fields and invalid values are rejected. Files must be at most 1 MiB and contain JSON-compatible values; duplicate keys and a stored confirm field are refused. The CLI does not expand environment variables or evaluate code in configuration files.

For a trial, save trial.yaml and replace the project, version and GPU with IDs returned by the list commands:

project_id: PROJECT_ID
dataset_id: VERSION_ID
gpu: NVIDIA GeForce RTX 4090
provider: runpod
kind: trial
objective: sft
method: lora
epochs: 2
learning_rate: 0.0002
rank: 32
alpha: 64
context_length: 2048
batch_size: 1
gradient_accumulation: 4
max_hours: 1
max_hourly_price: 1

tensorant training start --config trial.yaml --preview
tensorant training start --config trial.yaml

The second command displays a new preview and asks before spending. For a full run, set kind: full and add baseline_id for a completed baseline on the same version. For DPO, set objective: dpo and supply preference-pair data. The default learning rate, when omitted, is 0.0000005 for DPO and 0.0002 for SFT; an explicit value in a copied file overrides that default. See DPO requirements.

An explicit --project overrides a configuration file's project_id. A saved default project fills it only when the file and flag omit it. Version, run and evaluation IDs are explicit; the CLI does not silently select the latest result.

For a baseline against an existing endpoint, save baseline.yaml:

dataset_id: VERSION_ID
endpoint: ENDPOINT_NAME
primary_metric: exact_match
max_tokens: 256

tensorant evaluations start --config baseline.yaml --preview
tensorant evaluations start --config baseline.yaml --yes --wait

ENDPOINT_NAME is the name of a configured connection, or managed:DEPLOYMENT_ID for a ready test deployment. To evaluate a trained run, use a configuration containing run_id; its baseline determines scoring settings. Evaluation makes billed requests to the endpoint and, when configured, the judge.

Durable experiments

An experiment composes baseline evaluation, training, test deployment, tuned evaluation and cleanup. It runs on Tensorant, independent of your CLI process. Create a draft with a native experiment configuration; select its project through --project or the saved default. The configuration has nested training, serving, scoring and thresholds objects:

name: Support quality comparison
dataset_id: VERSION_ID
model_revision: main
automatic: true
training:
  provider: runpod
  gpu: NVIDIA GeForce RTX 4090
  objective: sft
  method: lora
  max_hours: 1
  max_hourly_price: 1
serving:
  gpu: NVIDIA GeForce RTX 4090
  max_hours: 0.5
  idle_minutes: 5
  max_hourly_price: 1
scoring:
  preset: qa
primary_metric: exact_match
max_tokens: 256
thresholds:
  minimum_score: 0.7
  minimum_delta_pp: 1
  minimum_groups: 10
  require_positive_interval: true
keep_serving: false

tensorant experiments create --project PROJECT_ID --config experiment.yaml
tensorant experiments start EXPERIMENT_ID --maximum-compute-estimate-usd 3 --preview
tensorant experiments start EXPERIMENT_ID --maximum-compute-estimate-usd 3 --yes
tensorant experiments watch EXPERIMENT_ID

Creating the draft requires Full access but rents no GPU. The draft records readiness checks, configuration and price estimates. Starting returns those saved limits and warnings in a spending preview before the backend begins or continues the workflow. Address readiness warnings, or explicitly accept them with --accept-warnings after reviewing their implications. Serving stages use RunPod even when training uses another connected provider.

Use automatic: false for a workflow that pauses at the supported approval points. Inspect the paused experiment, then use experiments start to continue through another spending preview. Watching a paused experiment returns a nonzero status so CI does not mistake a pause for completion. See Experiments for stages, scoring, cleanup and recovery.

The experiment's minimum_delta_pp uses percentage points: 1 means one percentage point. The CLI comparison flag --min-delta uses score fractions: 0.01 means the same one percentage point.

Spending controls

• --preview returns the server's spending preview without launching. Training checks readiness and prices; experiment previews show the draft's saved checks and limits, with prices enforced again by the backend at each launch stage. A preview does not save approval for a later command.
• An interactive launch displays a preview and asks for confirmation. A noninteractive launch requires --yes; JSON output alone is not consent to spend.
• The CLI obtains a new, short-lived confirmation for the exact settings on each launch and never includes it in output or configuration files. The server rechecks the request before acting.
• Training max_hours limits runtime, and max_hourly_price prevents a launch above the permitted hourly GPU price. Trials apply their shorter server-side trial limit.
• --maximum-compute-estimate-usd is the approved compute ceiling for an experiment. It must cover the maximum computed from the draft's saved time and hourly-price limits; a lower amount is refused even if the ordinary cost estimate is lower. Omit it to approve that required maximum after the preview. Retain the per-stage limits; endpoint, storage and other provider charges can be separate, and the preview is not a guarantee about the final provider bill.

If a launch or upload loses its response, inspect the project's runs, sources or experiments before retrying. A request may have succeeded even when the client saw a network failure. The CLI does not automatically replay mutations after a transport failure.

Pin the CLI in CI

Use the version-specific installer when a job should install a chosen CLI release:

curl -fsSL https://tensorant.io/downloads/cli/0.1.0/install.sh | bash -s -- --no-modify-path

It verifies the same embedded wheel checksum as the main installer. This pins the CLI wheel version; it does not pin every dependency. The website includes its current release and does not guarantee that older release URLs remain available after a future deployment. For long-lived pipelines, retain the rendered installer and matching wheel in your own artifact storage, or install the archived wheel directly with uv tool install.

The installer can provision uv and Python, so the runner does not need a separate Python setup step. Choose a tool directory and add it to the runner's PATH explicitly. For GitHub Actions:

- name: Install Tensorant CLI 0.1.0
  env:
    UV_TOOL_BIN_DIR: ${{ runner.temp }}/tensorant-bin
    UV_TOOL_DIR: ${{ runner.temp }}/tensorant-tools
  run: |
    curl -fsSL https://tensorant.io/downloads/cli/0.1.0/install.sh | bash -s -- --no-modify-path
    printf '%s\n' "$UV_TOOL_BIN_DIR" >> "$GITHUB_PATH"

Authenticate separately by injecting a token into only the steps that need Tensorant access. The installer does not need a token.

CI quality gates

Store a Read-level personal token as your runner's TENSORANT_TOKEN secret. Compare two completed evaluations of the same frozen version:

tensorant --json evaluations compare "$BASELINE_ID" "$TUNED_ID" \
  --min-delta 0.01 --require-confidence

The gate uses the evaluation's primary metric on paired examples. It passes only when their improvement is at least 0.01. With --require-confidence, the comparison must also provide a valid interval with at least 95% confidence whose lower bound is positive and meets that same threshold. Too little data for an interval, missing or non-finite scores, no paired examples, unfinished evaluations and different versions all fail the gate. A gate failure exits with code 4 and reports its reasons.

Without either gate flag, evaluations compare reports the comparison without asserting that it meets a quality threshold. Experiment completion alone also does not mean its acceptance thresholds passed; inspect the result or run a comparison gate. These checks describe performance on the held-out examples, not a guarantee for unseen requests.

For GitHub Actions, an example step after installing the CLI is:

- name: Check model quality
  env:
    TENSORANT_TOKEN: ${{ secrets.TENSORANT_TOKEN }}
    BASELINE_ID: ${{ vars.BASELINE_ID }}
    TUNED_ID: ${{ vars.TUNED_ID }}
  run: >-
    tensorant --json evaluations compare "$BASELINE_ID" "$TUNED_ID"
    --min-delta 0.01 --require-confidence

Use environment variables for external IDs rather than substituting them into shell source. A comparison gate only reads existing results; an automation that launches evaluation needs Build access, and training or experiments need Full access with --yes and explicit spending limits.

Output and exit codes

With --json, a successful command writes a JSON object containing data and summary to standard output. Errors write an object containing error and code. Status updates, spending prompts and progress go to standard error, so redirected standard output remains parseable. Watch timeouts and failed terminal states can include their final data with a nonzero exit code; always check the process exit status.

Wait and watch commands poll every 30 seconds by default, with a one-hour timeout. Use --poll-interval of at least 30 seconds and a positive --timeout, both in seconds. Closing a terminal, interrupting a watch or reaching its timeout does not cancel the operation. Cancel it explicitly with its job, run or experiment command when needed.