Qwen
Fine-tune Qwen3.8 Flash-Next on your data
Qwen's large mixture-of-experts model that reads images and text, trained as an experiment on its text with LoRA on one machine of 8 NVIDIA B200 GPUs. Experimental: few training steps of this setup have run at full size, so start with a trial.
- Status
- Experimental
- Input
- Text and images
- Kind
- Mixture of experts
- Trains with
- LoRA
- Suggested sizes
- LoRA from8 x NVIDIA B200
Sizes
Sizes and GPUs
Each size with the GPUs one machine needs to train and to serve it. The figures are the estimates Tensorant checks a model against; a trial measures the real use. See GPU memory.
| Model | Parameters | Train with LoRA | Train with QLoRA | Serve | Status |
|---|---|---|---|---|---|
| Qwen/Qwen3.8-Flash-NextMixture of experts; trains on 8 B200 (experimental) | 180B | 8 x NVIDIA B200about 134 GB each | - | 4 or 8 GPUsabout 438 GB in all | Experimental |
- Training needs 8 NVIDIA B200 GPUs of one machine, and the memory shown on each.
- Serve: the numbers of GPUs of one machine the base model can be split across, and the memory it needs in all.
Training
How it trains
Train Qwen3.8 Flash-Next with LoRA, by supervised fine-tuning (SFT).
- It trains on one machine of 8 NVIDIA B200 GPUs, its experts split across them: about 134 GB on each. Another GPU or number of GPUs is refused.
- It trains as an experiment, in one way only: SFT with LoRA on its text. A version with images, DPO, RL and QLoRA are refused.
- The adapter trains its attention layers, not its experts.
Every training run, trials included, pays a training-token fee for the tokens it trains on, by the model's size. Images are on every organization.
Serving
How it serves
Publish a trained model under a name and call it through the OpenAI-compatible API, with the OpenAI SDK or any HTTP client. The name is what your application sends as model.
- A LoRA adapter you train serves on top of its base model, on the GPUs in the Serve column.
- The base model serves on its own too, in a test session or for a baseline. A deployment splits it across the GPUs of one machine: the Serve column lists the numbers each size allows.
- A served vision model reads images and text in the same chat request.
- A trained adapter serves as the base model does, with images and text, a little more slowly than the base model alone.
- It runs in bfloat16 only, with a bfloat16 KV cache: leave its weight precision at Auto and its KV cache precision the same as the model's.
Steps
Fine-tune Qwen3.8 Flash-Next on Tensorant
Create a project
Create a project with
Qwen/Qwen3.8-Flash-Nextas its base model. The model check reads the model before anything starts.Import your data
Upload files, paste text or import a Hugging Face dataset as sources. It trains on text examples only.
Review and freeze a version
Review the examples and approve the ones to train on, then freeze them into a version with training, validation and test splits.
Score the base model
Evaluate the base model on the version's test split. This baseline is what a training run is compared with, and a full run needs one.
Run a trial, then train
A trial trains a small sample to check the settings, the GPU memory and the speed. Then start the full run. LoRA on
Qwen/Qwen3.8-Flash-Nextneeds 8 x NVIDIA B200, about 134 GB each.Compare and publish
Evaluate the trained model on the same test split and compare it with the baseline. Then publish it and call it through the OpenAI-compatible API.
Docs
Supported models
Every family's architecture, status and notes, and what a base model needs.
Model family
Qwen3.5 and Qwen3.6 MoE
Qwen's mixture-of-experts models that read images and text, with linear attention in most of their layers.
Model family
Qwen3.5, Qwen3.6 and Qwen3.8
Qwen's dense models that read images and text, with linear attention in most of their layers and long reasoning turned on by default.
Start with one dataset.
Create an account and import your data, or book a demo and we will show you the product.