Building Robust RAG: Evaluation-Driven Development with Phoenix and Giskard

In the early days of software engineering, we learned that untested code is broken code. We adopted Test-Driven Development (TDD) to ensure our logic met requirements before we even wrote the implementation. However, as we move into the era of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG), traditional unit tests—where an input x always yields a deterministic output y—are no longer sufficient.

LLMs are probabilistic, and RAG pipelines are complex. Between vector database retrieval, prompt templates, and the stochastic nature of the model itself, there are dozens of places where a system can fail silently. This is where Evaluation-Driven Development (EDD) comes in. EDD is the evolution of TDD for the AI age: a methodology where we define our evaluation metrics and quality gates before refining our RAG architecture.

In this guide, we will explore how to implement a production-grade EDD workflow using two powerhouse open-source tools: Arize Phoenix for observability and evaluation, and Giskard for automated vulnerability scanning and unit testing.

The Problem: Why Traditional Testing Fails RAG

In a standard web application, a unit test checks if a function correctly calculates a discount or saves a user to a database. In a RAG system, your 'unit' is often a combination of a natural language query, a retrieved set of documents, and a generated response.

A RAG system can fail in three distinct ways:

1. Retrieval Failure: The system fetches irrelevant documents.
2. Faithfulness Failure (Hallucination): The model generates an answer that isn't supported by the retrieved context.
3. Relevance Failure: The model answers the question, but the answer is unhelpful or ignores user constraints.

You cannot write a regex to catch a hallucination. You need a more sophisticated approach that uses 'LLM-as-a-judge' and automated scanning to quantify quality.

The Architecture of Evaluation-Driven Development

EDD isn't just about running a script before a commit; it’s a lifecycle. We start by capturing traces of our system in action, then we apply evaluation prompts to those traces, and finally, we codify those evaluations into automated test suites.

Arize Phoenix: The Observability Bedrock

Arize Phoenix provides a local-first observability platform that allows you to trace your LLM applications using OpenInference standards. It’s essential for EDD because you cannot evaluate what you cannot see. Phoenix allows us to look inside the 'black box' of a LangChain or LlamaIndex execution and see exactly what context was retrieved and how the prompt was constructed.

Giskard: The Automated Quality Gate

Giskard acts as the QA engineer for your LLM. While Phoenix helps us observe and run ad-hoc evaluations, Giskard allows us to perform 'scans' for common vulnerabilities (like prompt injection or hallucinations) and generate a suite of automated unit tests that can be integrated into a CI/CD pipeline.

Implementing the RAG Evaluation Triad

To build a high-quality RAG system, we focus on the "RAG Triad." Let's look at how to implement this using Phoenix and Giskard.

1. Context Relevance

Did our vector search actually find the right information? If your retrieval is noisy, your LLM will likely produce garbage.

Using Arize Phoenix, we can run a Relevance evaluator. This evaluator takes the user query and the retrieved chunks, then asks a stronger model (like GPT-4o) to rate the relevance.

from phoenix.evals import RELEVANCE_PROMPT_TEMPLATE, OpenAiModel, run_evals

# Define the evaluator model
eval_model = OpenAiModel(model="gpt-4o")

# Run the evaluation on your retrieved spans
relevance_df = run_evals(
    dataframe=my_trace_dataframe,
    descriptors=[RELEVANCE_PROMPT_TEMPLATE],
    model=eval_model,
    provide_explanation=True
)

2. Faithfulness (Hallucination Control)

This is the most critical metric for enterprise RAG. We compare the generated answer against only the retrieved context. If the answer contains facts not present in the context, it's a hallucination.

3. Answer Relevance

Does the response actually address the user's intent? A faithful answer that doesn't answer the question is still a failure.

Step-by-Step: Building the Automated Test Suite

Now, let's look at how to combine these tools into a repeatable workflow.

Step 1: Instrumentation and Tracing

First, we instrument our application. If you are using LangChain, this is as simple as setting an environment variable or using a callback. Phoenix will start capturing every step of the RAG process.

import phoenix as px
from phoenix.trace.langchain import LangChainInstrumentor

# Start the Phoenix server
session = px.launch_app()

# Instrument your LangChain app
LangChainInstrumentor().instrument()

Step 2: Running a Giskard Scan

Once we have a functional RAG pipeline, we use Giskard to perform an automated scan. Giskard will probe the model for hallucinations, harmful content, and performance issues by generating synthetic inputs.

import giskard

# Wrap your RAG function
def model_predict(df):
    return [rag_chain.invoke(question) for question in df["question"]]

giskard_model = giskard.Model(
    model=model_predict,
    model_type="text_generation",
    name="RAG_Support_Bot",
    description="Answers customer questions based on internal documentation",
    feature_names=["question"]
)

# Perform the scan
report = giskard.scan(giskard_model)

This scan produces a comprehensive report identifying exactly where the model is prone to hallucinating. It's the equivalent of a static analysis tool for LLMs.

Step 3: Codifying Evaluations into Unit Tests

From the Giskard scan, we can extract specific failing cases and turn them into a test suite. This is the core of EDD: when a hallucination is found, it becomes a permanent test case.

test_suite = report.generate_test_suite("My RAG Test Suite")

# Add a specific hallucination test
from giskard.testing import test_llm_faithfulness

test_suite.add_test(test_llm_faithfulness(giskard_model, dataset))

# Run the suite
results = test_suite.run()

Handling Hallucinations with LLM-as-a-Judge

A common challenge in EDD is the "Who evaluates the evaluator?" problem. Both Phoenix and Giskard leverage LLM-as-a-judge patterns. To make this reliable, we should follow these best practices:

1. Use a Stronger Model for Evaluation: If your RAG uses GPT-3.5 or a local Llama-3-8B, use GPT-4o or Claude 3.5 Sonnet as your evaluator.
2. Provide Chain-of-Thought (CoT): Always ask the evaluator to provide an explanation. Phoenix’s provide_explanation=True flag is vital here. It allows you to debug why a test failed.
3. Few-Shot Examples: Provide your evaluators with examples of what constitutes a 'good' vs. 'bad' response in your specific domain (e.g., medical vs. legal).

Integrating EDD into CI/CD

To make EDD effective, it must be part of your deployment pipeline. You wouldn't merge a PR that breaks your unit tests; you shouldn't merge a RAG prompt change that increases the hallucination rate.

1. The Golden Dataset: Maintain a 'Golden Dataset' of question-context-answer triplets that have been human-verified.
2. The Threshold Gate: Set a threshold for your metrics (e.g., Faithfulness > 0.95). If a change to the prompt or the chunking strategy drops this metric, the CI build fails.
3. Regression Testing: Every time a user reports a hallucination in production, add that query to your Giskard test suite to ensure it never happens again.

Advanced Pattern: Synthetic Data Generation

One of the biggest hurdles in EDD is the lack of labeled data. Giskard excels here by using an LLM to generate 'adversarial' questions based on your knowledge base. It can create questions that are intentionally ambiguous or based on common misconceptions in your data to see if your RAG system can handle them. This allows you to test your system against thousands of scenarios before a single real user interacts with it.

Conclusion: The Path to Production-Grade AI

Moving a RAG system from a local notebook to a production environment is a significant engineering challenge. The stochastic nature of LLMs requires us to rethink our quality assurance processes. By adopting Evaluation-Driven Development, we treat LLM quality as a first-class citizen.

Actionable Next Steps:

• Instrument Today: Add Arize Phoenix to your development environment to start seeing your traces.
• Define Your Triad: Choose your evaluation metrics. At a minimum, start with Faithfulness (hallucination detection).
• Automate the Scan: Run Giskard against your current RAG pipeline to identify low-hanging fruit and vulnerabilities.
• Build a Golden Dataset: Start curating 20-50 high-quality question-answer pairs to serve as your regression suite.

By shifting left and focusing on evaluation early, you transform your AI development from a game of 'vibe-checking' prompts to a disciplined, measurable engineering practice.