How can I compare a model across various hardware?
Easy, Deci runs it for you automatically. All you have to do is
- Click the model name under the Lab. This will lead you to the Insights page with the model’s comparisons.
- You can then hover over the Insights menu and click Benchmark HW.
- On the right side of the top bar, select the hardware and batch sizes you want to compare. You’ll get all the measurements under three different graphs: latency, throughput, and memory footprint.
Updated over 1 year ago