Vision models

A vision model reads images and text and answers in text. Use one when the questions are about pictures: charts, screenshots, scanned forms, receipts or product photos. The workflow is the same as for a text model, with images carried along: import examples with images, review, freeze a version, train, evaluate and serve. Your application then sends images through the API.

Only images are supported, not audio or video, and answers are always text.

Supported models

Like any base model, it must be public, ungated and have a chat template (see Projects). Its repository must also have a preprocessor_config.json and must not need custom code (no auto_map in its config files). The readiness checks before training or deploying verify this.

• Google's own Gemma 3 repositories are gated. Use an ungated copy such as unsloth/gemma-3-4b-it. Gemma 3 1B reads text only.
• SmolVLM2 is not supported. Use a SmolVLM version 1 model.
• Other vision models, such as Qwen3-VL mixture-of-experts models, Llama 3.2 Vision, LLaVA, Mistral Small 3.1 or Phi-3.5-vision, are not supported.

Before you start

• The Pro plan, for images: importing them, training and evaluating on them, and sending them to a model in the Playground or the API. On any plan, a vision model can be deployed, published and used with text. See Plans and billing.
• A project whose base model is one of the supported models.
• A RunPod network volume as your organization's storage, because images are kept there. RunPod bills the volume for the space. See Connections.
• An image dataset on Hugging Face. Uploaded files and pastes can't carry images.

From dataset to API

1. Create a project with a vision base model.
2. Import an image dataset from Hugging Face. See below.
3. Review the examples. Images show as thumbnails; you edit the text around them. See Review.
4. Freeze a version. The same image always stays in one split. See Versions.
5. Run a baseline: deploy the base model and evaluate it. Test examples with images can only be sent to your own deployments. See Evaluations.
6. Train an adapter with LoRA. See Training runs.
7. Deploy and evaluate the adapter against the baseline.
8. Try it in the Playground, publish it, and call it from your application.

An experiment runs steps 5 to 7 for you. Choose LoRA as its Training method.

Import images from Hugging Face

1. In Sources, choose Add sources, then Hugging Face.
2. Enter the dataset ID, choose Load dataset, then the configuration, split and starting row.
3. Choose Preview rows. Image cells show as their size, such as "Image · 850 × 600".
4. Under Import as, choose Prompt and answer examples or Chat / ShareGPT conversations, and set Image column (optional). In a vision project the suggested mapping fills it in for you.
5. Set Maximum rows and Assign to split, and choose Import up to N rows.

The import runs in Activity like any Hugging Face import.

The image column must hold an image or a list of images, up to 4 per row. They are placed like this:

• Prompt and answer examples: the images first, then the question.
• Chat / ShareGPT conversations: the images fill the conversation's placeholders in order, either {"type": "image"} parts or <image> markers in user messages. If the counts differ, the row is skipped. Without placeholders, the images go at the start of the first user message.

Only user messages can carry images. A row with an empty image cell becomes a text example.

How images are read

The import reads the original image files from Hugging Face's parquet export of the split, not the re-encoded copies in the preview. Each image is checked:

• PNG, JPEG, WebP and GIF are kept as they are. BMP and TIFF are converted to PNG. Other formats are refused.
• An image can be at most 40 megapixels and 20 MB.

A row with an image that fails a check is skipped, with the reason in the import report.

Where images are kept

Images are stored on your organization's RunPod network volume, in the images/ folder, with small thumbnails in thumbnails/. An image used in many examples, imports or projects is stored once.

Images pass through Tensorant when you import them, view them, or send them in the Playground or the API, but are not kept there. Training and evaluations read them straight from your volume, inside your RunPod account. Deleting an import does not delete its images, and Tensorant never deletes them: remove them in RunPod if you need to.

Images and the context length

Each image becomes tokens that count toward the context length, in training (Context length) and when serving (Serving context length). Approximate counts:

With Qwen models, the default training Context length of 2,048 is often too short for large photos. Raise it, and the deployment's Serving context length, to fit your longest example with its images and answer.

What is not supported

Limits

When something goes wrong

Choosing the model

Importing

Training, evaluating and serving