Create AI evaluations
Learn how to create evals, so your team can ship AI features with confidence.
Go back
Introduction to AI Evals
An introduction to AI Evals: why we need them, and how to create them.
What you'll learn
What to expect from this series, and what you should know before you start.
Mental model
Mapping your web testing knowledge to the world of large language models.
Design evaluations
Define what good and bad looks like for your AI application.
Build rule-based evaluations
Automate the basics. Use code to catch simple errors.
Build a basic judge, part 1
Get your subjective evaluations running with a basic judge model.
Build a basic judge, part 2
Finish setting up your basic judge model to get your subjective evaluations running.
Course resources
A non-exhaustive list of sources used in this course and eval tools that can help you.