JSON extraction
Fine-tune an LLM for JSON extraction and structured output
Fine-tune an open model to pull fields out of invoices, forms and emails as JSON that matches your schema, and score it field by field.
The problem
Why does extraction need a fine-tuned model?
A general model asked for JSON can name a field one way on one request and another way on the next. It can also put the JSON inside a sentence or a code block, and the code that parses the answer then fails.
Your code needs the same fields, with the same names and types, on every document. A model fine-tuned on your documents and the JSON you expect from each is trained to answer that way, and you can measure how often it does before you publish it.
How it works
How Tensorant trains and checks JSON
- A Structured JSON recipe writes examples from your documents, each answered with JSON for your schema. The schema is JSON Schema draft 2020-12, written inline, without
$ref,patternorformat. In Review, an example it wrote can be approved only when its answer matches the schema. - Structured JSON scoring records JSON-schema correctness, whether each answer is valid JSON that matches the schema. For the fields you list as paths, such as
/total, it also records JSON field accuracy, the share of those fields whose value matches the reference. - When you call the published model,
response_formatwith a JSON schema makes it answer in JSON that matches the schema. An answer cut off atmax_tokenscan hold incomplete JSON, so check thatfinish_reasonisstop. See Response format.
Models
Models to start with
Start with a small instruct model that answers directly. Every family Tensorant trains is in Supported models.
Qwen/Qwen3-4B-Instruct-2507
A 4B instruct model that answers directly, without a reasoning block, so its answer can be the JSON alone.
mistralai/Mistral-7B-Instruct-v0.3
A larger model to try if a 4B model misses fields on your test set.
unsloth/Llama-3.2-1B-Instruct
An ungated copy of Llama 3.2 1B Instruct. It needs the least GPU memory of these three.
Steps
How to do it in Tensorant
Create a project
Choose a base model and the Extraction to JSON use case. The project then offers the Structured JSON recipe and Structured JSON scoring first.
Import or generate examples
Import examples you already have, each a document's text as the question and the JSON you expect as the answer, in
questionandanswercolumns or as conversations. See Sources. To write examples from your documents instead, save a Structured JSON recipe with your schema and generate examples with a model endpoint or a deployment.Review against the schema
Approve the examples in Review, and fix or reject any answer that doesn't match the schema. Then create a version, which holds its test examples out for scoring.
Run an experiment
An experiment scores the base model, trains an adapter and scores the adapter on the same test examples. Score it as Structured JSON, with your schema and the fields to compare.
Publish it and call it
Give the adapter a name in API models and send requests with
response_formatset to your JSON schema. See the API quickstart.
Plans
What you need
- Text fine-tuning, evaluations and publishing are on every organization, with the same limits for all. Each training run pays a training-token fee from credits. See Limits on every organization.
- GPUs come from Tensorant compute, charged per second from prepaid credits, which an owner adds under Settings, Plan and billing (the first payment turns Tensorant compute on), or from a GPU account you connect with the Own compute subscription. See connections.
- Generating examples with a recipe sends your documents to the model endpoint you choose, and its provider bills you for the requests. See what is sent to the endpoint.
Case studies
Case study: fine-tuned 7B models beat GPT-4 on narrow tasks
How small LoRA-tuned open models outscored GPT-4 in the LoRA Land and Anyscale studies, and how to run that same comparison in Tensorant.
Case studies
Case study: a fine-tuned 7B model beats GPT-4o at text-to-SQL
How fine-tuned Qwen2.5-Coder 7B models outperformed GPT-4o on Spider and BIRD benchmarks, and how to test your own SQL model in Tensorant.
Evaluation
How to evaluate a fine-tuned LLM against its base model
Build useful LLM evals in Tensorant, score a support-ticket classifier, compare base and tuned models, and investigate regressions before deployment.
Private and sensitive data
Medical, legal and financial records: what Tensorant stores, where it runs, and what can stay in your cloud.
Classification and routing
Answer with one label from your list: intents, ticket routes, what needs a person.
Replace a costly LLM API
Train an open model on the answers your API model gives, and switch when it scores as well.
Small language models
Fine-tune a 1B to 4B model for one narrow job, and download the adapter.
Scoring and ranking
Score each candidate answer with a calibrated probability, with a decision model such as kev.
Reinforcement learning
Train with GRPO against a reward that checks each answer: a label, a number, JSON or your own code.
Migrate from OpenAI fine-tuning
OpenAI ends fine-tuning in January 2027. Move your data, train an open model and compare the two.
Start with one dataset.
Create an account and import your data, or book a demo and we will show you the product.