Classification and field extraction: JSON mode

Pull the amount, date, and counterparty out of an email; sort tickets into categories; extract the fields from an invoice: classic tasks where a model replaces brittle regular expressions. For this to work reliably, it must answer in strictly structured JSON.

response_format instead of "return JSON"

Asking "answer in JSON" in the prompt text works every other time: the model wraps it in ```json one moment and adds a comment on top the next. Use the parameter:

resp = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[
        {"role": "system", "content": "Extract the fields from the invoice. Return JSON."},
        {"role": "user", "content": invoice_text},
    ],
    response_format={"type": "json_object"},
    temperature=0,
)
data = json.loads(resp.choices[0].message.content)

With json_object the model returns JSON without wrappers. temperature=0 removes variability, which is exactly what you want for extraction.

A strict schema

If the model and the API support JSON Schema, lock the structure down completely. Then you will not have to check for every field by hand:

schema = {
    "type": "object",
    "properties": {
        "supplier": {"type": "string"},
        "amount": {"type": "number"},
        "currency": {"type": "string", "enum": ["RUB", "USD", "EUR"]},
        "date": {"type": "string", "description": "ISO 8601"},
    },
    "required": ["supplier", "amount", "currency", "date"],
    "additionalProperties": False,
}

enum is especially useful for classification: with a strict schema the model will not invent a category outside the list.

Classification: close the options

For labeling tickets, do not leave the model any freedom; give it a closed list of labels:

system = """Classify the request into one of the categories:
billing | technical | account | other.
If unsure, use other. Return JSON: {"category": "...", "confidence": 0.0-1.0}."""

The confidence field gives you a threshold for manual review: send everything below 0.7 to a human.

Choosing a model

• Simple extraction and classification: Flash models cope and cost pennies. Run them in batches.
• Complex documents (multi-page contracts, nested structure): a stronger model with a larger context.
• Vision is needed when the data is in a scan or a picture rather than in text; then you need a model that understands both text and images (more on that separately).

Pitfalls

• Numbers as strings. The model returns "amount": "1 000,50". A schema with "type": "number" makes it normalize, but check: it sometimes gets the Russian format with a space and a comma wrong. Ask for a dot and no thousands separators.
• Hallucinated fields. If a field is not in the document, the model may invent it. Instruct it: "a missing field is null, do not guess".
• null versus an empty string. Agree in advance and write it into the prompt, otherwise you get a mix.
• Retries. Even with json_object, broken JSON arrives once in a while. Wrap json.loads in a try and retry the request once: cheaper than complicating the parser.

Build a test set of 30 to 50 real documents with reference fields and run it whenever you change the model or the prompt. Extraction is a case where the accuracy metric is easy to compute and easy to improve.