Skip to content

DeepSeek

Fine-tune DeepSeek R1 distills on your data

Qwen and Llama models that DeepSeek trained on DeepSeek R1's answers, so they reason before they answer. Supported by the training and serving software Tensorant runs. It hasn't run on Tensorant yet, so start with a trial.

Status
Supported
Input
Text
Kind
Dense
Trains with
LoRA, QLoRA
Suggested sizes
LoRA from1 GPU, about 8 GB

Sizes

Sizes and GPUs

Each size with the GPUs one machine needs to train and to serve it. The figures are the estimates Tensorant checks a model against; a trial measures the real use. See GPU memory.

DeepSeek R1 distills models and the GPUs each needs
ModelParametersTrain with LoRATrain with QLoRAServeStatus
deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5BReasons before answering1.5B1 GPUabout 8 GB1 GPUabout 5 GB1, 2 or 4 GPUsabout 6 GB in allSupported
deepseek-ai/DeepSeek-R1-Distill-Qwen-7BReasons before answering7B1 GPUabout 22 GB1 GPUabout 9 GB1, 2 or 4 GPUsabout 20 GB in allSupported
deepseek-ai/DeepSeek-R1-Distill-Llama-8BReasons before answering8B1 GPUabout 23 GB1 GPUabout 9 GB1, 2, 4 or 8 GPUsabout 21 GB in allSupported
deepseek-ai/DeepSeek-R1-0528-Qwen3-8BReasons before answering8B1 GPUabout 24 GB1 GPUabout 9 GB1, 2, 4 or 8 GPUsabout 22 GB in allSupported
deepseek-ai/DeepSeek-R1-Distill-Qwen-14BReasons before answering14B1 GPUabout 40 GB1 GPUabout 14 GB1, 2, 4 or 8 GPUsabout 38 GB in allSupported
deepseek-ai/DeepSeek-R1-Distill-Qwen-32BReasons; LoRA needs a 96 GB GPU32B1 GPUabout 83 GB1 GPUabout 25 GB1, 2, 4 or 8 GPUsabout 81 GB in allSupported
deepseek-ai/DeepSeek-R1-Distill-Llama-70BReasons; LoRA needs a B20070B1 GPUabout 175 GB1 GPUabout 50 GB1, 2, 4 or 8 GPUsabout 173 GB in allSupported
  • Training needs the memory shown on each GPU: more GPUs on a machine train faster, but each holds the whole model.
  • Serve: the numbers of GPUs of one machine the base model can be split across, and the memory it needs in all.

Training

How it trains

Train DeepSeek R1 distills with LoRA or QLoRA, by supervised fine-tuning (SFT), DPO on preference pairs or reinforcement learning (RL).

  • RL trains with LoRA, on one GPU.
  • They reason before they answer, and their chat template drops that reasoning from the examples a run trains on: write your examples' answers as final answers.

Every training run, trials included, pays a training-token fee for the tokens it trains on, by the model's size.

Serving

How it serves

Publish a trained model under a name and call it through the OpenAI-compatible API, with the OpenAI SDK or any HTTP client. The name is what your application sends as model.

  • A LoRA adapter you train serves on top of its base model, on the GPUs in the Serve column.
  • The base model serves on its own too, in a test session or for a baseline. A deployment splits it across the GPUs of one machine: the Serve column lists the numbers each size allows.
  • An answer holds the model's reasoning, up to </think>, and then the answer. Tensorant doesn't remove the reasoning, in evaluations or through the API.

Steps

Fine-tune DeepSeek R1 distills on Tensorant

  1. Create a project

    Create a project with deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B as its base model, or pick another size from the list. The model check reads the model before anything starts.

  2. Import your data

    Upload files, paste text or import a Hugging Face dataset as sources.

  3. Review and freeze a version

    Review the examples and approve the ones to train on, then freeze them into a version with training, validation and test splits.

  4. Score the base model

    Evaluate the base model on the version's test split. This baseline is what a training run is compared with, and a full run needs one.

  5. Run a trial, then train

    A trial trains a small sample to check the settings, the GPU memory and the speed. Then start the full run. LoRA on deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B needs 1 GPU, about 8 GB.

  6. Compare and publish

    Evaluate the trained model on the same test split and compare it with the baseline. Then publish it and call it through the OpenAI-compatible API.

Related

See every model

Start with one dataset.

Create an account and import your data, or book a demo and we will show you the product.