LEIP Package

The goal of the LEIP Package is to enable deployment of executables provided by the LEIP compiler. This tool provides multiple ways of deploying the compiled Neural Network (NN) model because the end user may want to use different targets and use cases.

Overview

You do not need the LEIP SDK container to run the model once the model has been compiled. The LEIP framework currently supports the following ways to perform inference on a compiled model:

Through LEIP Evaluate or LEIP Run, which have limited support for detection based models;
Through a series of Python code examples provided in the examples directory of the Docker image that hosts the LEIP framework; and
Through a C++ Wrapper or C Wrapper, which are meant for embedded design and thus produce a set of artifacts along with a Makefile that requires user additions before generating an executable.

The LEIP Package is a Latent AI Runtime Environment (LRE) Object that can be accessed by the end user through an API. The LRE Object encapsulates a number of services, such as authentication, encryption, watermarking, and pre/post processing that enables the end user to build an application. By default, no services are added into the LRE Object. This results in a small memory footprint.

CLI Usage

The basic command is:

CODE

$ leip package --input_path /my_optimized_model --metrics accuracy,most_common_class --preprocessor imagenet --postprocessor is_person --output_path /my_self_contained_lre_python_package

Format

You can specify the target language of the generated LRE Object using --format. The options are Python and C++. The C++option generates a directory structure with a Makefile and libraries and includes headers and the C++ source code.

Metrics

You can specify the metrics to be collected at runtime using --metrics. The valid options are: accuracy, latency, and most_common_class.

Pre and post processing

The pre and post processing can be added to the LRE Object using --preprocessor and --postprocessor respectively. By default, no pre nor post processing services are added into the LRE Object. This results in a small memory footprint.

For a list of the valid pre and post processors supported by the LEIP SDK please consult the CLI Reference for LEIP Evaluate.