Ragas Dataset Converter: RAGAS ↔ TruLens ↔ LangSmith JSON

What this tool does

RAG evaluation frameworks store test datasets in incompatible shapes. Ragas uses columnar parallel arrays, TruLens uses one record per line item, and LangSmith nests everything under inputs / outputs. This converter detects which format you pasted, converts it to the target format, and tells you exactly which fields were renamed, restructured or dropped — so nothing disappears silently.

• Convert — Ragas (legacy ground_truth and current reference), TruLens and LangSmith, in any direction.
• Field changes — every renamed, flattened, nested or dropped field, listed explicitly.
• Validation — unequal parallel arrays, empty questions, and contexts stored as a plain string are flagged with their record index.

FAQ

What format does a Ragas evaluation dataset need to be in?

Ragas expects a columnar JSON object with parallel arrays: question, contexts, answer, and reference (called ground_truth in older versions). All arrays must be the same length, and each contexts entry must be a list of strings, not a plain string:

{
  "question": ["What is RAG?"],
  "contexts": [["Retrieval-augmented generation combines..."]],
  "answer": ["RAG combines retrieval with generation."],
  "reference": ["RAG retrieves documents and generates answers."]
}

How do I convert my Ragas dataset to TruLens?

Paste your Ragas JSON above, select TruLens, and copy the result. Each sample becomes a record with query, expected_response and expected_chunks. The chunks keep their original order — order matters because retrieval metrics depend on it. Note that TruLens has no reference field, so that column is dropped and reported in the field-changes list.

Why does Ragas say "ground_truth" is not a valid field?

Ragas renamed ground_truth to reference in newer versions. Datasets created for the old schema fail validation with this error. Rename the field to reference — this converter does it automatically when you select RAGAS current — and the dataset loads again.