A BENCH IS NOT A BUILDING.

Evaluation in robotics is usually conducted where evaluation is easiest. A test rig, a fixed camera position, controlled lighting, an uncluttered surface, one object at a time, no people moving through the space.

There are good reasons for this. Controlled conditions are repeatable, which makes results comparable between runs and between models. Without repeatability there is no measurement at all.

But repeatable is not the same as representative, and the gap between the two is where reliability is lost.

A live operating environment introduces, routinely and without warning:

• Reflective and damaged packaging that defeats depth sensing
• Targets partly obscured by other objects, or obscured at the moment of grasp
• Lighting that varies by hour, by fixture and by weather
• People crossing the workspace, moving the target, requiring the robot to yield
• Interruption, and objectives that change mid-task
• Surfaces that are wet, greasy, compliant or uneven
• Sensors that accumulate contamination and drift out of calibration

None of these is exotic. Each is ordinary in a warehouse, a kitchen, a shop floor or a hospital corridor. And each represents a condition that a controlled evaluation, by design, removes.

A benchmark result reports performance at one point in condition space. Usually the most favourable point available, because that is the point at which measurement is easiest.

Consider a policy reporting 92 per cent on a standard evaluation. That number is real. It may also conceal that performance holds above 85 per cent across seven conditions and falls below 40 per cent on partial occlusion. The single figure averages the collapse away.

The buyer's question is never "is this model good". It is "under which of my conditions does it stop working, and how does it fail when it does". A score cannot answer that. A profile can.

The distinction between failing safely and failing unsafely is the one an operator asks about first, and it is absent from almost every published benchmark.

The alternative is not to abandon controlled testing. Baseline measurement under controlled conditions is necessary, because without it there is nothing to compare degradation against.

The alternative is to continue past the baseline. Establish performance under controlled conditions, then raise one condition at a time and record where the curve bends. Then combine the conditions that produced the steepest degradation, because deployment presents them together rather than one at a time. Then induce failure deliberately and record whether the system detects it, re-attempts, and recovers.

And conduct as much of that as the deployment allows in the environment itself, because a working kitchen produces steam, clutter, wet surfaces and human traffic in combinations that no test rig reproduces convincingly.

It changes what evaluation is for.

Evaluated as a score, a policy is either good enough or it is not, and there is nothing to do with the result except decide.

Evaluated as a profile, every failure is specific and every specific failure is addressable. A policy that breaks on reflective surfaces under partial occlusion is telling you precisely which data its training never contained. The evaluation becomes a specification for what to collect next, and the improvement can then be measured against the same conditions that exposed the weakness.

That is the loop worth building: test where the work happens, find what breaks, identify what the training data was missing, build it, and test again.

The evidence that current data does not contain these conditions is set out in what the data actually contains. The nine conditions, their grading, and the evaluation protocol are specified in the Physicore Standard v0.1.