RAGAS Dataset Format: Every Field, With Examples
The modern SingleTurnSample schema, the legacy question/contexts/answer schema, and the fields TestsetGenerator adds.
Field changes
- —
Validation warnings
- —
What you can evaluate
- —
Two RAGAS schemas, one file
RAGAS has used two field naming schemes. Older tutorials and datasets use question, contexts, answer and ground_truth. The current SingleTurnSample uses user_input, retrieved_contexts, reference_contexts, response and reference. When a pandas DataFrame is exported to JSON, both look like one object of parallel arrays.
Modern schema (current RAGAS)
| Field | Meaning | Required |
|---|---|---|
user_input | The user's question | Yes |
retrieved_contexts | Chunks your pipeline actually retrieved | For retrieval metrics |
reference_contexts | Gold chunks a test question was built from | Optional |
response | The answer your pipeline generated | For answer metrics |
reference | The expected answer (ground truth) | For reference-based metrics |
retrieved_context_ids, reference_context_ids | Chunk identifiers for retrieval analysis | Optional |
multi_responses, rubrics | Multiple candidate answers, per-sample grading rubric | Optional |
persona_name, query_style, query_length | Synthetic test set metadata | Optional |
synthesizer_name | Which TestsetGenerator synthesizer produced the row | Optional |
{
"user_input": ["What is RAG?"],
"retrieved_contexts": [["RAG combines retrieval with generation."]],
"reference_contexts": [["Retrieval-augmented generation retrieves documents and generates answers."]],
"response": ["RAG retrieves documents, then generates an answer."],
"reference": ["Retrieval-augmented generation retrieves relevant documents and uses them to generate answers."],
"synthesizer_name": ["single_hop_specific_query_synthesizer"]
}
Legacy schema
The legacy schema still appears in older blog posts and datasets. It is the same columnar shape with different names:
{
"question": ["What is RAG?"],
"contexts": [["RAG combines retrieval with generation."]],
"answer": ["RAG retrieves documents, then generates an answer."],
"ground_truth": ["Retrieval-augmented generation retrieves relevant documents and uses them to generate answers."]
}
ground_truth was renamed to reference. If your code expects reference and the file still says ground_truth, validation fails.
Fields added by TestsetGenerator
TestsetGenerator produces user_input, reference_contexts, reference and synthesizer_name (plus optional persona_name, query_style and query_length). A generated test set contains gold contexts, not your pipeline's retrieved contexts. Replace reference_contexts with retrieved_contexts before you run retrieval metrics, or the scores describe the generator, not your retriever.
Common format mistakes
contextsis a string. Each entry must be a list of strings:[["chunk one", "chunk two"]], not["chunk one"]used as a single string.- Unequal array lengths. Every parallel array must have one entry per sample. A shorter array shifts every later row onto the wrong record.
- Gold contexts in the wrong field.
reference_contextsis what the question was built from;retrieved_contextsis what your retriever returned. Context Recall compares the two. - CSV instead of JSON. Convert a DataFrame with
df.to_json(orient="records")ordf.to_dict(orient="list"). The converter accepts JSON and JSONL, not raw CSV.
Which fields each metric needs
Faithfulness needs user_input, retrieved_contexts and response. Context Recall and Context Precision also need reference. The full metric-by-metric breakdown is in RAGAS without ground truth.
Primary sources
Field names and metric definitions on this page follow the upstream projects:
- RAGAS documentation — datasets and sample schema — the upstream definition of EvaluationDataset and SingleTurnSample
- ragas/dataset_schema.py on GitHub — the source of truth for every field name listed above