RAG on Command A: answers with citations you can trust

In an enterprise assistant, getting an answer is not enough: you need to know whether it can be trusted and how to check it. Hence the requirement: every statement must rest on a specific fragment of a document, and that fragment must be visible. Command A is built for exactly this kind of work.

What RAG consists of

1. Indexing. We cut documents into chunks of 300 to 800 tokens, turn each one into a vector with an embedding model and store them in a vector store.
2. Retrieval. For the user's question we compute the query embedding and fetch the top-N nearest chunks.
3. Generation. We hand the chunks to Command A with a strict instruction to answer only from them and to cite the source.

The first two steps are about embeddings (we have a separate article on choosing them), the third is about the model.

Indexing the fragments

We compute vectors for the chunks via /v1/embeddings:

from openai import OpenAI
client = OpenAI(base_url="https://api.yenisei.ru/v1", api_key="YENISEI_KEY")
def embed(texts):
    r = client.embeddings.create(model="bge-m3", input=texts)
    return [d.embedding for d in r.data]
vectors = embed([chunk.text for chunk in chunks])
# store vectors in the vector DB along with the chunk id and metadata

Generating an answer grounded in sources

Here everything comes down to the system prompt and how the fragments are formatted. We give each chunk an explicit identifier so the model can reference it:

context = "\n\n".join(
    f"[{c.doc}#{c.id}] {c.text}" for c in retrieved
)
resp = client.chat.completions.create(
    model="command-a",
    messages=[
        {"role": "system", "content": (
            "Answer strictly from the fragments below. "
            "After every statement, add a reference in the format [doc#id]. "
            "If the answer is not in the fragments, write: 'The documents do not contain the answer.'"
        )},
        {"role": "user", "content": f"Question: {question}\n\nFragments:\n{context}"},
    ],
)
print(resp.choices[0].message.content)

Why Command A specifically

The main thing about it is discipline. A model without strict boundaries will, on a borderline question, pad the answer with "common sense" from its general knowledge, and in an enterprise assistant that is a mistake. Command A by its nature sticks to the attached sources better and says "I don't know" more honestly when the answer is not in the context. With a 256K window it takes many fragments at once, so on broad questions you do not have to sacrifice completeness.

What most often breaks RAG

• Bad chunking. Cut the document into 2000-token pieces and retrieval drags in chunks where the relevant part is diluted with noise. Cut too fine and context is lost. 300 to 800 tokens with a small overlap is usually the sweet spot.
• One embedding for everything. A Russian-language base must be indexed with a multilingual model (BGE-M3, Qwen3 Embedding), otherwise retrieval degrades.
• No fallback "I don't know". Without an explicit instruction the model will always answer, and on an off-topic question it will produce a plausible fabrication.
• Mixing roles. The "sources only" instruction belongs in the system message, and the fragments in the user message; if you pile everything together, discipline drops.

How to check quality

Collect thirty or so real questions with known answers and run the pipeline on them after every change: switched the embedding model, changed the chunk size, tweaked the prompt. The metric is simple: the share of answers where the statements are confirmed by the cited fragments. If a citation does not match the text, that is a false positive, and it is more dangerous than a plain "I don't know".