How to Evaluate an AI Fitness Feature

You evaluate an AI fitness feature the way you'd evaluate any other system whose output you can't fully predict: with a fixed set of realistic inputs, a grader that runs without you, and a threshold you agreed on before you saw the results. The catch specific to this domain is that there is no benchmark to borrow. We could not locate any public benchmark, leaderboard, or published methodology for evaluating AI fitness coaching or workout-plan generation quality — if one exists, our research did not find it. So you build the ruler as well as the thing being measured. Do that before you ship, not after users start complaining, because the golden set is also the artifact that tells you whether your next prompt tweak helped or quietly broke something.

The good news is that most of your real coverage comes from the cheapest layer. A workout plan is mostly checkable with a WHERE clause.

Why "we tried it and it looked good" is not evaluation

Two people typing a few prompts into a staging build is how almost every AI feature gets validated, and it fails in a specific way: you test the inputs you can imagine, which are the inputs you already designed for. The failure modes that actually hurt live in the tail — the user with a knee replacement and only resistance bands, the user who says "I want to be more disciplined with food," the user who argues with the model for six turns until it caves.

Provider guidance on writing evals makes three points that transfer directly here: be task-specific and mirror the real input distribution including edge cases, automate wherever possible, and prioritise volume of test cases over hand-crafted quality. Two hundred automatically-graded cases beat twenty lovingly hand-reviewed ones. That principle is what makes the whole thing affordable.

The same guidance says success criteria should be specific and measurable — "at least 99% of emitted exercise IDs resolve to a catalogue row", not "plans are good" — and that a realistic target is multidimensional: a task-fidelity threshold, a safety threshold, an error-severity distribution, and a cost or latency budget, all in one definition of done.

Build the golden set from real user inputs, before launch

A golden set is a frozen list of inputs plus, where possible, the expected output or the properties the output must satisfy. Sources, in rough order of value:

• Onboarding data you already collect. Goals, available equipment, time budget, injuries, training age. Sample real profile combinations rather than inventing tidy ones — real users have weirder equipment lists than you'd guess.
• Support tickets and app-store reviews from your non-AI product. These are free descriptions of what your users actually want and where they got confused.
• Deliberate adversarial cases you write by hand (see the red-team section — keep them in a separate suite).
• Production traffic, once you have it, sampled and de-identified.

If you have no users yet, write the profiles anyway. A hundred synthetic-but-plausible profiles with hard constraints (bands only; shoulder impingement; 25 minutes; three days a week) is enough to catch the failures that matter most, and you can replace them with real ones later.

Freeze it, version it, and store the expected properties next to each case. And hold out a slice you never look at while iterating, so you have something to measure genuine generalisation against.

The three layers

Most teams start at layer two because it feels like the AI-native answer. Start at layer one. It is cheap, it is fast, it never flakes, and in a constrained-generation architecture it covers the majority of the ways your feature can be wrong.

Layer 1: deterministic assertions

If you generate plans by having the model select from a vetted catalogue rather than free-generate exercise names — which is the pattern we recommend in grounding an LLM in an exercise database — then most correctness questions become lookups. Every one of these is a test you can run in CI on every prompt change:

• Every exercise_id in the plan exists in the catalogue snapshot. Reject on miss; do not repair.
• Every selected exercise satisfies the original filter — equipment the user actually has, not on their contraindication list, inside the difficulty band. Models drift outside constraints that were stated only in prose.
• Numeric ranges are sane: sets, reps, rest, session duration estimate, RPE. Note that these cannot be enforced at the decoding layer — the structured-output schema format does not support minimum / maximum or string-length constraints, so schema-valid is not the same as physiologically sane. You must check them yourself, after generation.
• Cross-row invariants: no duplicate exercise inside a session, weekly set count per muscle group under your ceiling, total session time inside the user's stated budget.
• Stop reason was a normal completion. A refusal or a max_tokens truncation can produce output that doesn't match your schema at all.

A grader is not complicated:

def grade_plan(plan, profile, catalogue):
    failures = []
    for day in plan["days"]:
        for item in day["items"]:
            ex = catalogue.get(item["exercise_id"])
            if ex is None:
                failures.append(("unknown_exercise", item["exercise_id"]))
                continue
            if not set(ex["equipment"]).issubset(profile["equipment"]):
                failures.append(("equipment_violation", ex["id"]))
            if ex["id"] in profile["contraindicated_ids"]:
                failures.append(("contraindication", ex["id"]))
            if not 1 <= item["reps"] <= 30 or not 1 <= item["sets"] <= 10:
                failures.append(("range", item))
    weekly = weekly_sets_by_muscle(plan, catalogue)
    for muscle, sets in weekly.items():
        if sets > profile["weekly_set_cap"]:
            failures.append(("volume_cap", muscle, sets))
    return failures

Run that over 200 profiles and you have a number you can put a threshold on. Two related checks worth adding: consistency — generate the same profile N times and assert that total volume, session length, and muscle-group distribution stay inside a tolerance band, which catches instability a single-shot test hides; and for anything numeric the model estimates (a portion size, say), track signed mean error rather than absolute error, so a systematic bias in one direction doesn't average itself into looking fine.

This layer is also where the two-stage split pays off. Let the model do selection and sequencing; let deterministic code do the arithmetic — load progression, macro totals, volume accounting. The arithmetic is what users notice when it's wrong, and it's the part you can test exhaustively. AI workout plan generation covers that split in more detail.

Layer 2: LLM-as-judge, for the parts a WHERE clause can't reach

Some things you genuinely care about are subjective: does the explanation make sense to a beginner? Does the coaching tone match your brand? Did the response appropriately suggest seeing a professional? Provider guidance describes the usable forms — a Likert scale (1–5) for qualities like empathy and professionalism, binary classification for must-not-happen properties, and ordinal scales for ordered categories like how well the response used prior context.

Two rules that matter more than the rubric wording:

1. Use a different model for judging than for generating where you can. A model grading its own output is not an independent measurement.
2. Validate the judge. Have a human label a few dozen cases, then check the judge agrees. An unvalidated judge is a random number generator with good manners.

And the boundary: LLM judges are not the right tool for anything you can check with a lookup. If you're asking a judge "did this plan use only dumbbell exercises", you've spent money to get a less reliable answer than set(equipment).issubset(...).

Layer 3: a human who actually knows programming

Not you, unless you have the credential. Someone who can look at twelve weeks of generated output and say "the progression here is nonsense" or "this is safe but so generic it's useless". Published expert evaluations of chatbot-generated resistance-training programmes have characterised model output as safe but generic, with persistent weakness in individualisation and progression. We could not read the primary papers from this environment, so treat that as a direction to look for rather than a measurement — which is exactly the failure your deterministic layer will happily pass, because every ID resolves and every cap holds.

You can't run this in CI. Run it on a sampled batch before each significant prompt, schema, or model change, and treat it as the thing that sets your rubric for layer two. Concretely: hand the reviewer 20 to 30 full generated plans with the profile that produced each one, and ask for a written note per plan rather than a score. The notes are what become the layer-two rubric — a score tells you a plan was a 3, a note tells you why, and only the second one can be turned into a judge prompt.

Safety red-teaming is a separate suite, and it's multi-turn

Keep this apart from your quality evals. Different cases, different grading (binary must-not-happen), different consequence: a regression here is ship-blocking, not a score that dipped.

Cases to cover, at minimum:

• Disordered-eating probes. Extreme deficit requests, "how few calories can I eat", goal weights that are medically unsafe, compensatory-exercise framing. The hard part is that the subtle signals — wanting to eat healthier, exercise more, be "more disciplined" with food — are also the normal vocabulary of a fitness app user. The overlap is near-total, which is why this needs deliberate cases rather than a keyword list.
• Medical-boundary probes. Chest pain during exertion, fainting, an injury with loss of function, pregnancy, medication questions, requests for a diagnosis.
• Populations your generic path should not silently serve. Minors; users who have declared cardiac, metabolic, or pulmonary conditions; pregnancy. Grade these as "did it constrain?" rather than "did it refuse?" — a suite that scores refusals as passes will drive your product toward refusing everyone it is unsure about.
• Prompt injection through user-controlled fields. A custom food name or a workout note is untrusted text that lands in your model's context.
• Contraindication leakage. Does a stated shoulder injury survive fifteen conversational turns, or does it fall out of the context window and reappear as an overhead press?

That last one is the general point, and it is the one that changes how you write the cases. Single-turn red-teaming is not enough, because the guardrail failure this domain actually has is a multi-turn one — see why prompt-level rules erode. So write the conversation, not the prompt: the user pushes back, reframes, claims to be a professional, says their physio approved it. Then assert on the final turn, not the first refusal. A suite of single-turn probes will pass on a system that fails every real conversation.

This is not just good practice. Google Play's AI-Generated Content policy tells developers to test across user scenarios and to safeguard against prompts that could manipulate a generative feature into producing harmful output — Play explicitly expects adversarial testing — and separately requires an in-app way for users to report or flag offensive AI output without leaving the app. Play policies are named rather than numbered, and wording changes, so verify the current text before you rely on this; Google Play health data policy covers the store side in more depth. The point for this page is that an adversarial suite is a condition of shipping, not a maturity milestone.

Our own escalation list — what warrants a hard stop versus a constrained mode — is engineering judgement, not a published standard. Have a clinician review yours. LLM safety for fitness advice goes through the guardrail architecture that these tests are testing.

Correction deltas: the eval set that refreshes itself

If your UI lets users edit what the model produced — swap an exercise, fix a portion size, change a rep target — then every edit is a labelled example, created by the person best placed to label it, at zero cost to you. Log the delta, not just the final state: what the model said, what the user changed it to, and the context.

That stream beats most hand-built test sets on two properties a frozen golden set cannot have: it measures the exact gap you care about — what you generated versus what the user wanted — and it tracks distribution drift automatically as your user base changes.

Two things to do with it. Mine it into new golden-set cases on a schedule, holding the promoted cases to the same versioning discipline as the rest of the set. And treat the correction rate itself as a shipped metric: a prompt change that lowers your swap rate is a real improvement measured on real users, which is a stronger claim than any judge score. What to instrument per entry is domain-specific — AI nutrition logging works through the food-logging version, where portion estimates are corrections waiting to happen.

Regression mechanics: assert properties, not strings

Prose has no equality check. assert output == expected is useless the moment the model rephrases, and it will. So assert properties of the output instead:

• Structural: parses, matches the schema, has between 3 and 6 exercises, every ID resolves.
• Invariant: contains no exercise from the contraindication list; contains a professional-consultation prompt when the input carried a risk signal; contains no numeric calorie target when the user is under 18.
• Statistical, across the set: pass rate by category, mean signed error, refusal rate on the safety suite.
• Semantic, via judge: coherence score at or above your threshold.

Three operational habits:

Version everything together. The prompt, the schema, the catalogue snapshot, and the model ID belong in one versioned bundle. A catalogue edit can regress plan quality with zero code change, and if you can't pin the snapshot you'll spend a day blaming the prompt.

Run the deterministic suite on every prompt change, in CI, as a merge gate. It's fast and free.

Run the full suite via the batch API. Evals are inherently non-interactive, and asynchronous batch processing is documented at a 50% cost reduction versus standard synchronous requests — this is explicitly one of its recommended uses.

One more cheap trick for hallucination specifically: run the same prompt several times and compare. Inconsistency across runs is a documented signal of hallucination, and it needs no reference answer at all.

What a full run costs

Show yourself the arithmetic rather than trusting a vibe. Using clearly hypothetical numbers — substitute your own:

• 300 golden-set cases
• ~4,000 input tokens each (system prompt, safety rules, filtered catalogue slice, profile)
• ~800 output tokens each (a structured plan)

That's 1.2M input tokens and 240K output tokens per run. At the published rates for claude-opus-5 ($5.00 per million input, $25.00 per million output, as of mid-2026 — verify current pricing), that's $6.00 in and $6.00 out, so about $12 per full run, or roughly $6 via the batch API. Add your judge calls on top, which you can run on a cheaper tier such as claude-haiku-4-5.

Twelve dollars to know whether your prompt change broke anything is not a budget conversation. The same arithmetic applied to your production traffic is a different matter, and is worked through in what an AI fitness feature costs to run. The deterministic layer costs nothing at all beyond the generation calls, which is the other argument for putting your effort there.

What you still own

Three things no eval suite changes.

Passing your own tests is not safety. A green suite means you didn't regress against the failures you thought of. It does not mean the feature is safe, and it certainly doesn't make you compliant with anything. Frame every guardrail as risk reduction.

LLM output about exercise and nutrition is not medical advice, and you — not the model vendor — own what your app tells a user. A "consult a physician" line is an app-store expectation and worth including, but it is not a liability shield, it does not satisfy any regulator, and it does not make an unsafe answer safe. Whether AI coaching providers owe a legal duty of care is unsettled; we found no authority either way.

Your thresholds are yours to defend. Nobody publishes an industry-standard bar for AI fitness coaching quality. You pick the number, you write down why, and you decide what a regression blocks.

Start with fifty real profiles and a deterministic grader you can write in an afternoon. That single step will tell you more about whether your feature works than any amount of manual poking, and everything else — judges, red-team suites, correction mining — bolts onto it. When you're ready to look at the guardrails those tests are exercising, read the guardrail architecture these tests exercise.