Deterministic Evals
Listed inDeterministic EvalsEvaluationon
Exact match, schema validation, and assertions that never disagree with themselves.
Intermediate6 minDraft
#methods
Knowing whether a change helped — the only thing separating engineering from vibes.
10 articles
Listed inDeterministic EvalsEvaluationon
Exact match, schema validation, and assertions that never disagree with themselves.
Listed inRagasEvaluationon
Faithfulness, answer relevance, and context precision — how the metrics are computed and what they miss.
Listed inLLM EvaluationsEvaluationon
The open-source eval framework and registry — a good template for structuring your own graded test set.
Listed inLLM EvaluationsEvaluationon
Building a test suite for a system that answers differently every time.
Listed inDeepEvalEvaluationon
Pytest-style evals you can run in CI.
Listed inRegression TestingEvaluationon
Catching quality drops when prompts, models, or data change.
Listed inModel-Based EvalsEvaluationon
LLM-as-judge, its biases, and how to calibrate it against humans.
Listed inEvaluation MetricsEvaluationon
Faithfulness, relevance, latency, cost — picking metrics that map to the product.
Listed inRagasEvaluationon
A RAG-specific eval framework for faithfulness and context precision.
Listed inHuman EvalsEvaluationon
Rubrics, annotator agreement, and when humans are the only ground truth.