Insights

Notes on evaluating AI systems before they meet real users.

A four-part series on choosing AI testing tools — written because “what’s the best eval tool?” is the wrong question.

• Note · 1 of 4

  There is no best AI testing tool

  A dashboard that scores prompts is not a red-team tool. Measurement is not assurance.

• Note · 2 of 4

  Name what you are testing before you pick a tool

  Model, prompt layer, retrieval, end-to-end, agent trajectory, production. Six different objects.

• Note · 3 of 4

  A practical map of AI testing tools

  The landscape by job to be done, not by marketing category — with shortlists you can actually start from.

• Note · 4 of 4

  How to choose without getting sold a dashboard

  Name the release decision, write the contract, calibrate the judges, then test the measurement system itself.