Select the BENCHMARK HW option to display graphs showing the benchmark metrics of the selected model for multiple target hardware environment and batch size at the same time.
The following displays --
For example, if Throughput is the most important goal of your model, this graph shows that optimal throughput is attained on V100 hardware (marked in purple) at a batch size of 32; and if Latency is the most important goal of your model, this graph shows that optimal latency is achieved on V100 hardware at a batch size of 1.
Note – The Model Cloud Cost graph is coming soon.
You can select one or more target hardware environments and batch sizes to appear in the graph by checking the checkboxes in the Select Hardware and the Select Batch Size dropdown menus.
Updated over 1 year ago