DocumentationAPI Reference
Back to ConsoleLog In

Loading a Model

LOAD MODEL – In a Python environment, load the model using the Infery infery.load command as follows –

import infery, numpy as np

model = infery.load(model_path='MyNewTask_1_0.onnx', framework_type='onnx', inference_hardware='gpu')
  • model_path – Specify the exact path to where you downloaded/saved your model.
  • framework_type – Specify the framework (programming language) used to develop this model. The supported options are listed in the table below.
  • inference_hardware – Specify either gpu or cpu according to the target hardware environment on which the model will be run (CPU or GPU).
  • static_batch_size – The static batch size the model graph is frozen for if the model is static, None if it is dynamic.
  • inference_dtype – The numpy data type to use for inference.

Loading Model Example:

# Downloading a pre-trained ResNet50 (Imagenet) model, that supports inference with batch size up to 64.
from urllib.request import urlretrieve

# Load it with infery
import infery
model = infery.load(weights_path='/tmp/model.onnx', framework_type='onnx')

If the model loaded successfully, you should see successful output logs –

__init__ -INFO- Infery was successfully imported with 2 CPUS and 1 GPUS.
infery_manager -INFO- Loading model /tmp/model.onnx to the GPU
infery_manager -INFO- Successfully loaded /tmp/model.onnx to the GPU.

If INFERY failed to load the model for any reason, you should be able to see the cause for the error with a verbose description –

infery_manager -ERROR- Failed to register model: The model file does not exist at /tmp/model.onnx
      1 import infery
----> 2 model = infery.load(model_path='/tmp/model.onnx', framework_type='onnx')
FileNotFoundError: The model file does not exist at /tmp/model.onnx

Did this page help you?