Smart-camera benchmark plan
A proposed evaluation using commercially available hardware. Targets will be agreed with a device partner and results will be recorded once testing is complete.
Compare against the current camera
- Agree on the baseline, sample scenes and acceptance targets before testing.
- Run both setups with matched resolution, lighting, motion patterns and standby-to-active duty cycle. Include day and night scenes where relevant.
- Repeat measurements, record firmware and model versions, and report typical results and variation.
Proposed benchmark criteria
Agree numerical thresholds with the device partner before testing. The table defines what the pilot will measure.
| Metric | Test method |
|---|---|
| Standby and deep sleep | Record whole-device standby power and deep-sleep current at the stated supply voltage. |
| Active power and energy | Record average active inference power and energy per wake-to-decision cycle, including sensor and host. Record radio energy separately. |
| Wake-to-decision latency | Report median and 95th-percentile time from the agreed trigger to the detection decision. |
| Detection quality | Measure missed detections and false alerts on labeled day and night scenes held out from model tuning. |
| Memory and integration | Record model storage, peak runtime RAM, board area, and estimated component cost at an agreed volume. |
Evaluation architecture
- Image sensor
- Inference on evaluation hardware
- Host MCU or camera firmware
- Wireless radio
When would custom silicon make sense?
Start with available microcontrollers and camera processors. If wake-up energy, sustained inference power, memory or component cost misses an agreed requirement, test whether a dedicated design could close the gap while maintaining detection quality.
Download the Smart-Camera Evaluation Brief (PDF)
Decision after the pilot
Compare results against the agreed thresholds and document any trade-offs. Proceed to chip feasibility only if the measurements and customer demand justify the design cost.