In step one, LEIP Recipes were used to export a pre-trained, traced model. The second step of the recipe compiles and compresses this model in the same Docker container to produce the binary artifact for the model optimized to the architecture of your target device. You will then be able to copy this artifact and the associated files to the target device for evaluation.

Step two of the recipe uses the LEIP Pipeline command with a predefined build configuration provided in /latentai/recipes to ensure the best compilation results.

If your hardware target is an Xavier NX or AGX, use the following commands to compile and optimize your model:

export PYTHONPATH=/latentai/recipes/yolov5_L_RT/evaluation/utils:$PYTHONPATH

# Pipeline build for ARM64 with Nvidia Volta or Turing GPUs
leip pipeline \
   --input_path /latentai/artifacts/recipe_yolov5l_batch1_640-640.pt \
   --output_path /latentai/workspace/recipes_output/yolov5_L_RT \
   --config_path /latentai/recipes/yolov5_L_RT/pipeline_aarch64_cuda.json
CODE

If you don’t have access to a target device and want to test the model within the SDK docker container, use the following commands to compile and optimize your model:

export PYTHONPATH=/latentai/recipes/yolov5_L_RT/evaluation/utils:$PYTHONPATH

# Pipeline build for x86_64 with Nvidia Volta or Turing GPUs
leip pipeline \
   --input_path /latentai/artifacts/recipe_yolov5l_batch1_640-640.pt \
   --output_path /latentai/workspace/recipes_output/yolov5_L_RT \
   --config_path /latentai/recipes/yolov5_L_RT/pipeline_x86_64_cuda.json
CODE

Regardless of which pipeline you choose, two flavors of the binary artifact will be created:

  • A Float32 version at: /latentai/workspace/recipes_output/yolov5_L_RT/Float32/01-compile/

  • An Int8 version at: /latentai/workspace/recipes_output/yolov5_L_RT/Int8/01-optimize/

Preparing the model files

Now that you have generated the model artifacts, you will need to copy them out of the Docker container in order to run the model on the target. There are several files and directories that you will need to copy over. A script has been provided to facilitate copying these files into a tar archive file:

In the Docker container:

# Gather the model artifacts to copy to the target
cd /latentai/recipes/yolov5_L_RT
sh ./create-artifacts-tar.sh

# An archive will be created at: /latentai/recipes/yolov5_L_RT/model-artifacts.tar.gz
CODE

Copying the dependency and model files

Before you can evaluate the model on the target device, you need to retrieve two archive files that will be copied to the AGX target. The first archive is a set of dependencies that you will need to install on the AGX the first time that you configure the target for testing.

You can skip these copy steps if you are going to test the model within the SDK docker container.

Note: The following examples assume you have named your running SDK Docker container leip. If you have used another name, please modify these Docker commands accordingly.

# Run this command on the host machine to copy the dependency file from the container
# You will later copy this file to the AGX target

docker cp leip:/agx-install.tar.gz .
CODE

Next, copy the model files that you prepared above with the create-artifacts-tar.sh script. You will need to repeat this step each time you have generated new model files:

# Copy the model artifacts from the docker container to the host

docker cp leip:/latentai/recipes/yolov5_L_RT/model-artifacts.tar.gz .
CODE

Next, you will evaluate your optimized model on the target device.