Measure the whole path
Inference time is only one part of the timing budget. Include image acquisition, conversion, pre-processing, queue delay and the controller interface. Evaluate timing on the selected device with the expected image size and workload.
Check conversion against a baseline
ONNX Runtime documents quantization approaches and accuracy considerations. Compare the converted or quantized model with the baseline on representative evaluation data. A smaller model is useful only if its decision quality remains suitable for the task.
Exercise device constraints
Run long enough to observe memory use, queue growth and thermal behavior. Check startup, camera interruption, network loss and power recovery. Record the hardware, software and model versions with the results.
Make unavailable results visible
Expose health, freshness and deadlines to the consuming application. Decide whether a missed result causes hold, review or another defined fallback. Keep a recovery path that operators can understand and engineering teams can diagnose.