RAGAS Without Ground Truth: Which Metrics Still Work

Paste your dataset and the readiness panel marks every RAGAS metric ready or missing.

Paste your dataset JSON
Target:
Converted output


        

Field changes

  • —

Validation warnings

  • —

What you can evaluate

  • —

Short answer

RAGAS does not always need ground truth. Faithfulness, Response Relevancy and Aspect Critic / Rubrics are reference-free. Context Recall, Context Entities Recall, Answer Correctness, Factual Correctness, Semantic Similarity and Noise Sensitivity need a reference. Context Precision is the one worth checking twice: the default ContextPrecision needs a reference, but RAGAS also ships LLMContextPrecisionWithoutReference, which scores chunks against the generated response instead.

Metric readiness table

MetricNeedsReference required?
Faithfulnessuser_input, retrieved_contexts, responseNo
Response Relevancyuser_input, responseNo
Aspect Critic / Rubricsuser_input, response, optional rubricsNo
Context Entities Recallreference, retrieved_contextsYes (reference answer, not gold chunks)
Context Precisionuser_input, retrieved_contexts, referenceYes by default — LLMContextPrecisionWithoutReference drops it
Context Recalluser_input, retrieved_contexts, referenceYes
Answer Correctnessuser_input, response, referenceYes
Factual Correctnessresponse, referenceYes
Semantic Similarityresponse, referenceYes
Noise Sensitivityuser_input, retrieved_contexts, response, referenceYes

What a reference-free dataset looks like

You need the question, the chunks your retriever returned, and the answer your generator produced. No reference column:

{
  "user_input": ["What is RAG?"],
  "retrieved_contexts": [["RAG combines retrieval with generation."]],
  "response": ["RAG retrieves documents, then generates an answer."]
}

Paste this into the tool above. The readiness panel marks Faithfulness, Response Relevancy and Aspect Critic as ready, and lists the metrics that still need a reference.

How to check your own dataset

  1. Paste the JSON, JSONL or a RAGAS TestsetGenerator export into the input panel.
  2. Read the What you can evaluate panel. Green metrics can run now; red metrics name the missing fields.
  3. If a metric you need is red, add the missing column. For reference-based metrics, fill reference with an expected answer.

If you need reference-based metrics

There are three practical options. Generate a synthetic test set with TestsetGenerator, which produces reference and reference_contexts. Label a sample of production questions by hand. Or start with reference-free metrics, find the failure clusters, and add references only for those cases. The last option is usually the cheapest way to get a useful first evaluation.

Why Context Recall needs a reference

Context Recall checks whether the retrieved chunks contain every claim in the expected answer. Without an expected answer there is nothing to decompose into claims, so the metric has no target. If you only have questions, generated answers and retrieved chunks, use Faithfulness and LLMContextPrecisionWithoutReference instead — both score the retrieved chunks against your own output, so neither needs a gold answer.

Primary sources

Field names and metric definitions on this page follow the upstream projects:

Related