RAGAS Without Ground Truth: Which Metrics Still Work
Paste your dataset and the readiness panel marks every RAGAS metric ready or missing.
Field changes
- —
Validation warnings
- —
What you can evaluate
- —
Short answer
RAGAS does not always need ground truth. Faithfulness, Response Relevancy and Aspect Critic / Rubrics are reference-free. Context Recall, Context Entities Recall, Answer Correctness, Factual Correctness, Semantic Similarity and Noise Sensitivity need a reference. Context Precision is the one worth checking twice: the default ContextPrecision needs a reference, but RAGAS also ships LLMContextPrecisionWithoutReference, which scores chunks against the generated response instead.
Metric readiness table
| Metric | Needs | Reference required? |
|---|---|---|
| Faithfulness | user_input, retrieved_contexts, response | No |
| Response Relevancy | user_input, response | No |
| Aspect Critic / Rubrics | user_input, response, optional rubrics | No |
| Context Entities Recall | reference, retrieved_contexts | Yes (reference answer, not gold chunks) |
| Context Precision | user_input, retrieved_contexts, reference | Yes by default — LLMContextPrecisionWithoutReference drops it |
| Context Recall | user_input, retrieved_contexts, reference | Yes |
| Answer Correctness | user_input, response, reference | Yes |
| Factual Correctness | response, reference | Yes |
| Semantic Similarity | response, reference | Yes |
| Noise Sensitivity | user_input, retrieved_contexts, response, reference | Yes |
What a reference-free dataset looks like
You need the question, the chunks your retriever returned, and the answer your generator produced. No reference column:
{
"user_input": ["What is RAG?"],
"retrieved_contexts": [["RAG combines retrieval with generation."]],
"response": ["RAG retrieves documents, then generates an answer."]
}
Paste this into the tool above. The readiness panel marks Faithfulness, Response Relevancy and Aspect Critic as ready, and lists the metrics that still need a reference.
How to check your own dataset
- Paste the JSON, JSONL or a RAGAS TestsetGenerator export into the input panel.
- Read the What you can evaluate panel. Green metrics can run now; red metrics name the missing fields.
- If a metric you need is red, add the missing column. For reference-based metrics, fill
referencewith an expected answer.
If you need reference-based metrics
There are three practical options. Generate a synthetic test set with TestsetGenerator, which produces reference and reference_contexts. Label a sample of production questions by hand. Or start with reference-free metrics, find the failure clusters, and add references only for those cases. The last option is usually the cheapest way to get a useful first evaluation.
Why Context Recall needs a reference
Context Recall checks whether the retrieved chunks contain every claim in the expected answer. Without an expected answer there is nothing to decompose into claims, so the metric has no target. If you only have questions, generated answers and retrieved chunks, use Faithfulness and LLMContextPrecisionWithoutReference instead — both score the retrieved chunks against your own output, so neither needs a gold answer.
Primary sources
Field names and metric definitions on this page follow the upstream projects:
- RAGAS documentation — available metrics — which metrics exist and what each one scores
- RAGAS documentation — Faithfulness — the reference-free metric used as the example above