RAGAS Dataset Format: Every Field, With Examples

The modern SingleTurnSample schema, the legacy question/contexts/answer schema, and the fields TestsetGenerator adds.

Paste your dataset JSON
Target:
Converted output


        

Field changes

  • —

Validation warnings

  • —

What you can evaluate

  • —

Two RAGAS schemas, one file

RAGAS has used two field naming schemes. Older tutorials and datasets use question, contexts, answer and ground_truth. The current SingleTurnSample uses user_input, retrieved_contexts, reference_contexts, response and reference. When a pandas DataFrame is exported to JSON, both look like one object of parallel arrays.

Modern schema (current RAGAS)

FieldMeaningRequired
user_inputThe user's questionYes
retrieved_contextsChunks your pipeline actually retrievedFor retrieval metrics
reference_contextsGold chunks a test question was built fromOptional
responseThe answer your pipeline generatedFor answer metrics
referenceThe expected answer (ground truth)For reference-based metrics
retrieved_context_ids, reference_context_idsChunk identifiers for retrieval analysisOptional
multi_responses, rubricsMultiple candidate answers, per-sample grading rubricOptional
persona_name, query_style, query_lengthSynthetic test set metadataOptional
synthesizer_nameWhich TestsetGenerator synthesizer produced the rowOptional
{
  "user_input": ["What is RAG?"],
  "retrieved_contexts": [["RAG combines retrieval with generation."]],
  "reference_contexts": [["Retrieval-augmented generation retrieves documents and generates answers."]],
  "response": ["RAG retrieves documents, then generates an answer."],
  "reference": ["Retrieval-augmented generation retrieves relevant documents and uses them to generate answers."],
  "synthesizer_name": ["single_hop_specific_query_synthesizer"]
}

Legacy schema

The legacy schema still appears in older blog posts and datasets. It is the same columnar shape with different names:

{
  "question": ["What is RAG?"],
  "contexts": [["RAG combines retrieval with generation."]],
  "answer": ["RAG retrieves documents, then generates an answer."],
  "ground_truth": ["Retrieval-augmented generation retrieves relevant documents and uses them to generate answers."]
}

ground_truth was renamed to reference. If your code expects reference and the file still says ground_truth, validation fails.

Fields added by TestsetGenerator

TestsetGenerator produces user_input, reference_contexts, reference and synthesizer_name (plus optional persona_name, query_style and query_length). A generated test set contains gold contexts, not your pipeline's retrieved contexts. Replace reference_contexts with retrieved_contexts before you run retrieval metrics, or the scores describe the generator, not your retriever.

Common format mistakes

  • contexts is a string. Each entry must be a list of strings: [["chunk one", "chunk two"]], not ["chunk one"] used as a single string.
  • Unequal array lengths. Every parallel array must have one entry per sample. A shorter array shifts every later row onto the wrong record.
  • Gold contexts in the wrong field. reference_contexts is what the question was built from; retrieved_contexts is what your retriever returned. Context Recall compares the two.
  • CSV instead of JSON. Convert a DataFrame with df.to_json(orient="records") or df.to_dict(orient="list"). The converter accepts JSON and JSONL, not raw CSV.

Which fields each metric needs

Faithfulness needs user_input, retrieved_contexts and response. Context Recall and Context Precision also need reference. The full metric-by-metric breakdown is in RAGAS without ground truth.

Primary sources

Field names and metric definitions on this page follow the upstream projects:

Related