Skip to content

Deci Inference Examples

A collection of demo-apps and inference scripts for various frameworks using infery.

infery is Deci's Inference Engine for Python with robust API for inference and benchmarks, on any hardware.

Basic Examples (Copy-Paste scripts)

Advanced Examples

Application Examples

Custom Hardware Examples

Getting Started

  • Infery
  • You can use the Docker Container for GPU installations.
  • Or, Install on your host machine
  • Git LFS
    • We use Git LFS to store the example models checkpoints.
  # 1. Install dependencies: 
  python3 -m pip install infery[all]  #  or < infery-gpu[all] | infery[openvino] | infery-gpu[tensorrt_8.0] | ... > depending on your usage
  apt-get install git-lfs

  # 2. Clone this repo and download the example models
  git clone && cd infery-examples/ && git lfs fetch && git lfs checkout

  # 3. cd to the example's directory and run an example (The ONNX one, in this case).
  cd basic && python3

Infery Documentation


  • Feel free to request a feature, example or application.
  • Open an issue on GitHub and describe your desired usage.
  • Please format the code before opening a pull-request using ./scripts/

Support Matrix

PyTorch TensorFlow 2 OnnxRuntime OpenVino TensorRT CoreML TorchScript TFLite Keras
Supported Versions torch 1.10.1 tensorflow ≥ 2.6.0 OnnxRunntime ≥ 1.10.* openvino ≥ 2022.1.0 TensorRT coremltools ≥ 5.2.0 torch 1.10.1 tensorflow ≥ 2.6.0, or tflite-runtime tensorflow ≥ 2.6.0
CPU Support ✅ (*) ✅ (*)
GPU Support ✅ (*) ✅ (*) ✅ (*)
Model Predict - Synchronous
Model Predict - Asynchronous
Model Benchmark
Detect Model Input Dimensions Automatically ❌ - manual ❌ - manual
Detect Model Batch Size Automatically ❌ - manual ❌ - manual
Detect Model Input Data Type Automatically ❌ - manual ❌ - manual
Dynamic Batch Size
Open In Netron
Multiple Inputs \ Outputs ✅ (**)
Dynamic Dimensions (in addition to Batch Size)
Change Model Input Dimensions, Batch Size and Data Type
Model Meta-Data (Legend) Basic Basic Basic Basic Basic + TRT specific fields Basic Basic Basic Basic
Per-Layer Inference Profiling
Benchmark Modes synchronous synchronous synchronous synchronous synchronous with IO, synchronous without IO, Asynchronous synchronous synchronous synchronous synchronous
  • (*) will always use GPU if infery-gpu installed
  • (**) Limited multi-output support