Testing AI, not just using AI to test
This page is about testing software that uses AI — an LLM-powered feature, chatbot, or autonomous agent your team has built. That's a different problem from using AI to test software, which is what our posts on agentic QA in CI/CD and AI QA agents cutting regression time cover. Both matter, but they're separate disciplines: one is a tool in our regression pipeline, the other is the product under test here.
Eval sets built from real usage, not a handful of happy-path prompts
An LLM feature doesn't fail the way a deterministic one does — the same input can produce different acceptable outputs, and the failure modes (a plausible-sounding wrong answer, a dropped instruction, a tool called with the wrong argument) rarely show up in a quick manual check. Our approach starts with an eval set built from real user scenarios — the actual range of inputs, edge cases, and adversarial prompts your feature will see — rather than a small set of prompts someone wrote in an afternoon. Deterministic, code-based checks run first wherever a rule can grade the output; LLM-as-a-judge is reserved for the qualitative slice a rule genuinely can't score.
LLM-as-a-judge, calibrated against human review
A judge model grading another model's output is only useful if it's been checked against real human judgment first. Our approach to that calibration:
- Narrow, binary criteria. Judge prompts ask specific yes/no questions rather than an open-ended quality score — judges are far more reliable on "did the answer cite the source document" than on "was this a good answer."
- Human-reviewed disagreements. Outputs the judge scores as failing or as unclear are routed to human review, and the judge's agreement rate with that human baseline is tracked over time, not assumed once and left unchecked.
- Bias-aware grading. Known judge failure modes — favoring longer answers, favoring whichever answer is shown first — are controlled for with randomized ordering and, where it matters, a judge model from a different provider than the one being graded.
Agents need step-level testing, not just a final-answer check
A single agent task can involve dozens of model calls — planning, tool selection, tool-call arguments, intermediate reasoning — before a final answer appears, and a pass/fail check on that final answer alone can miss a wrong tool call that happened to still land on a correct-looking output. Our approach grades the steps that matter for your workflow: was the right tool chosen, were the arguments valid, did a multi-step task recover from a failed step instead of silently giving up. We also check the guardrail layer directly — refusal behavior, hallucinated facts or citations, and whether the agent stays inside the boundaries your product defines — rather than inferring it from the final output alone.
Handling non-determinism without pretending it isn't there
The same prompt can produce a different output on two separate runs — sampling temperature, model updates, and tool latency all contribute. Testing a single run against a fixed expected string treats a feature that's actually working as if it had failed. Our approach runs each case multiple times and compares an averaged pass rate against a baseline within a stated tolerance, rather than relying on one run as a verdict either way.
Regression runs whenever the prompt or model changes
Prompts and model versions change often, and each change is a release that can regress behavior just like a code change can. Our approach treats them that way: prompts are version-controlled, the eval suite runs before a prompt revision or model upgrade ships, and the rendered prompt itself is snapshotted and diffed against a committed baseline so an accidental wording change from an unrelated refactor gets caught before it reaches production.
How this fits an engagement
AI/LLM testing is delivered the same way as our other services — engineers embedded in your team or running testing end to end, on your own hours or the 24-hour cycle. If you're not sure whether this is the starting point you need, the free Release Readiness Assessment can point you to the right one.