DocumentationAPI Reference
Back to ConsoleLog In

Lab Tab – Optimizing Your Models

The Deci Lab is your deep learning model repository and is where you can manage, optimize and deploy your models. It shows both the baseline models that you uploaded into the Deci platform, as well as the various optimized versions that you created from this baseline model in the Deci platform.

The Deci Lab is displayed by default when you launch Deci or by clicking the Lab tab at the navigation bar.

Model List

Each model that you upload into the Deci platform is called a baseline model and is listed as a row. Baseline models are the untouched, unoptimized version of the model that you uploaded into the Deci platform.

Underneath each baseline model and slightly indented from it are each of the optimized versions of that baseline model.

By default, the Lab is provided with a pre-loaded well-known deep learning model architecture (ResNet50), intended for image-based classification, trained over the ImageNet dataset. The ResNet50 model is presented below as an example baseline model in order to demonstrate the process of optimizing a model using the Deci platform. If you are a new user, the top row of the Lab shows this baseline ResNet50 model, and below it, you will see its optimized versions.

The naming and measures of the optimized models listed in the Optimized Versions section may differ from those presented in this example.

The Lab’s Model List shows the following data for each model from left to right –

Color Indicator

The leftmost side of each row shows a color indicator. Each model is automatically designated its own color and all its optimized versions are represented by that same color throughout the Deci platform.

Optimization Status Icon

An icon appears to represent the status of each model, as follows –

  • AutoNAC Optimization™ – Indicates that this model has already been optimized using Deci AI’s proprietary Automated Neural Architecture Construction (AutoNAC) engine. AutoNAC redesigns your deep learning models in order to provide dramatically increased throughput, significant reductions in inference latency and substantial cost-to-serve savings. These are often accompanied by improvements in accuracy. This is Deci’s most powerful optimization feature and provides up to a 10X performance boost and up to 80% cost-savings, without compromising the model’s trained accuracy. To enjoy the full power of Deci’s AutoNAC accelerator, you must purchase a premium license. See AutoNAC for more information.
  • Optimized – Indicates that this model has already been optimized using the free option provided with the Deci platform.
  • In Progress – Indicates that the optimization process of this model is in progress.
  • Not Optimized (Baseline) – Indicates that this is the unoptimized baseline model that was uploaded into the Deci platform.

Version Number

Each baseline model that is uploaded is assigned a number in the format of #.#.
The leftmost digit is a sequential number of all the uploaded baseline models.
The next digit is a sequential number of the optimize versions of each baseline model.
For example, 1.3 represents the third optimized version of the first uploaded baseline model.

Model Name and Description

Next to the model’s name is a small icon that you can click to display the description of the model that was entered when the model was added/edited.

Accuracy

The accuracy of each model is presented in the Accuracy column, so that you can see what the original accuracy of this model was (before Deci optimized it – in the top row) and then the accuracy of each of the Optimized Versions created by Deci (in the rows indented underneath it).

Note: Currently, Deci does not validate the accuracy of the baseline model. It simply displays the accuracy that was declared by the user when the model uploading.

In the example below, we see that the unoptimized ResNet50 model has an Accuracy of 76.10% (Top-1) that was measured while it performed image classification on an ImageNet Dataset. The Lab shows that Deci’s optimized models have a very similar accuracy on each of two production target hardware environments –

  • 76.30% on T4.
  • 75.98% on Intel Xeon CPU.

The Deci platform’s objective is to provide the highest score (described below) without compromising accuracy at all or only slightly (within a given statistical error of 1%). Accuracy may even be improved slightly, as in the example shown above for a T4 environment.

Deci Score

Deci assigns a standardized/normalized score to describe the efficiency of a model’s runtime performance on a specific production environment, without compromising accuracy. A Deci Score is a normalized grade of between 1 and 10 that evaluates a model’s runtime efficiency on a specific hardware and batch size set up. It is comprised of performance metrics and measures for Accuracy, Throughput, Latency, Memory Footprint and Model Size. See Score for more details.

In the example above, we see that the unoptimized ResNet50 model has a Score of 7.0 for the primary environment and batch-size for which it was configured. As you can see, after Deci optimized the models, each has a significantly superior score –

  • 9.6 in a T4 production environment
  • 9.4 in an Intel Xeon production environment

Target HW

The Target HW column of the ResNet50 model specifies the primary environment on which this baseline model was run before optimization. For each optimized version of this model, this column shows the target hardware environment for which it was compiled.

Note: Deci automatically benchmarks each model version for efficiency on all supported CPU and GPU target hardware environments. Deci also automatically benchmarks for all available batch sizes on each target hardware. For CPU-optimized models, it benchmarks the batch size selected by the user; and for GPU-optimized model, it benchmarks up to the batch size selected by the user on the specified hardware.

In the example above –

  • The baseline ResNet50 model was run on a T4 environment.
  • One optimized version created by Deci, named T4-Optimized v1.1, was created to run on the same environment, meaning on a T4 environment.
  • The second Deci-optimize version, named T4-Optimized v1.2, was created to run on an Intel Xeon environment.

Quantization

Quantization is a method for cutting down a neural network to a reasonable size, while still achieving high-performance accuracy. This process maps a large set of input values to a smaller relevant set of output values, thus dramatically reducing the memory requirements and computational cost of using neural networks.

  • For uploaded models (before Deci optimization), this column shows 32 bit, meaning that Deci assumes a quantization level of 32 bits for the baseline model.
  • For optimized model versions, this column shows the quantization level of the optimization.

Task

Each model can handle only a single task, which is specified in this column, such as Classification, Semantic Classification, Object Detection, Depth Estimation or Pose Estimation.

Dataset

Specifies the name of the dataset on which this model was trained. The optimized versions of the model are usually trained on the same data set as the baseline model that you uploaded, unless the dataset was changed or updated, which then requires the optimized model to be fine-tuned within the Deci platform.

Owner

Specifies the Deci user who uploaded or optimized a specific model.

Created

For baseline models, specifies the date when this model was first uploaded to the Deci platform.
For optimized models, specifies the date on which the optimization was completed.

Model Action Buttons

Selecting the row of a baseline or an optimized model displays various buttons on the right that are applied to the selected model –

  • Edit – Enables you to edit all the Deci platform parameters that were defined for this model when the model was added/edited. If you change the INPUT DIMENSION of this model, the platform will automatically rerun a benchmark on this model in order to provide a more accurate result.
  • DeployDeploys the RTiC inference engine Docker.
  • Delete – You can delete either an unoptimized baseline model or an optimized model. Deleting an unoptimized baseline model also deletes all its optimized versions.
  • OptimizeEnables you to optimize an uploaded model. Optimization accelerates model inference performance by 2x – 10x on any hardware, without compromising accuracy, by using Deci’s Automated Neural Architecture Construction (AutoNAC) technology.
  • InsightsEnables you to benchmark your model. Benchmarking is a critical part of optimizing the inference performance of your model. It enables you to compare a model’s efficiency according to your goals and objectives. Deci measures and displays performance changes across different target hardware environments and batch sizes.

Quick Review of a Model’s Performance Metrics in the Lab

In the Lab, you can hover over the radar graph icon (shown below) to display the details of a model’s runtime performance as reflected by the metrics that determine its Deci Score.

Hovering this icon displays a Deci radar graph that shows the batch size and hardware for which this model was optimized and the absolute value of each metric of the model’s score – Accuracy, Throughput, Latency, Memory Footprint and Model Size , which was measured on the configured primary/target hardware and batch-size:

See Viewing Benchmark Insights for a full description of how to see Deci's insights and to benchmark your model in Deci.

Lab Summary

The top left of the Lab summarizes the quantities of the models that it contains, as follows –

  • Total Models – Specifies the quantity of all the models in the Lab, including optimized and not optimized (baseline).
  • Optimized – Specifies the quantity of all the optimized models in the Lab.
  • In Progress – Specifies the quantity of all the models currently in the midst of being optimized.
  • Not Optimized – Specifies the quantity of all the baseline (unoptimized) models in the lab.

Lab Action Buttons

The top right of the Lab provides various action buttons for handling the models in the Lab. Simply, select a model’s row and then click one of the following buttons –


Did this page help you?