Benchmarking is a critical part of optimizing the inference performance of your model. It enables you to compare a model’s efficiency according to your goals and objectives. Deci measures and displays performance changes across different target hardware environments and batch sizes by selecting them from the dropdown menus described below.
The Deci Insights tab uses Deci's Runtime Inference Container (RTiC) to send requests to your model on various target hardware environments while measuring CPU/GPU memory and computational utilization.
Open the Deci Lab.
Click on a model in order to investigate and gain insights into its performance.
The Insights page displays, as shown below –
Updated 7 months ago