Traditional software testing methods are ineffective when applied to AI-driven chatbots, retrieval-augmented generation (RAG) systems, and agents. Unlike deterministic systems, large language models (LLMs) generate probabilistic responses, which makes it difficult to verify correctness, consistency, and relevance. In this session, Mario-Leander Reimer will explore how to test for truthfulness, bias, hallucinations, and real-world usability, moving beyond theory to demonstrate practical testing strategies.

Using Testkube and a combination of popular AI testing frameworks, Mario-Leander Reimer will show attendees how to design automated tests and build an agentic AI testbench to continuously assess the performance of AI workloads across different scenarios. Whether attendees are developing a customer support bot, a RAG-powered assistant, or an autonomous agent, this presentation will provide them with the necessary tools to ensure that their AI workloads are reliable and perform well in practice.