8.1.1.1.1.8. blueoil.cmd.measure_latency

8.1.1.1.1.8.1. Module Contents

8.1.1.1.1.8.1.1. Functions

_pre_process(raw_image, pre_processor, data_format)

_measure_time(config, restore_path, step_size)

_run(config_file, experiment_id, restore_path, image_size, step_size, cpu)

run(config_file, experiment_id, restore_path, image_size, step_size, cpu)

main(config_file, experiment_id, restore_path, image_size, step_size, cpu)

Measure the average latency of certain model’s prediction at runtime.

blueoil.cmd.measure_latency._pre_process(raw_image, pre_processor, data_format)
blueoil.cmd.measure_latency._measure_time(config, restore_path, step_size)
blueoil.cmd.measure_latency._run(config_file, experiment_id, restore_path, image_size, step_size, cpu)
blueoil.cmd.measure_latency.run(config_file, experiment_id, restore_path, image_size, step_size, cpu)
blueoil.cmd.measure_latency.main(config_file, experiment_id, restore_path, image_size, step_size, cpu)

Measure the average latency of certain model’s prediction at runtime.

The latency is averaged over number of repeated executions – by default is to run it 100 times. Each execution is measured after tensorflow is already initialized and both model and images are loaded. Batch size is always 1.

Measure two types latency, First is overall (including pre-post-processing which is being executed on CPU), Second is network-only (model inference, excluding pre-post-processing).