Release Notes

September 5, 2025 - LEIP Deploy 2.1

Enhanced Automatic EP Selection: LEIP Deploy now automatically selects the optimal Execution Provider (EP) for your target hardware when none is specified, improving deployment efficiency and performance.
CUDA Graph Integration: Inference with CUDA Graphs is now supported. For static models, CUDA Graph is enabled automatically, resulting in faster and more consistent inference.
CUDA Streams for Multi-Model Inference: LEIP Deploy now leverages CUDA streams to enable concurrent inference across multiple models, maximizing GPU utilization and throughput.

License Key Format Changes: Users can now place their LEIP license keys in ~/.config/leip/license.key. If you prefer to use an environment variable, set it using LEIP_LICENSE_KEY. Note that LICENSE_KEY is now deprecated.
Simplified Sign-up Process: New users can now receive a one-year trial license by visiting https://accounts.latentai.io/sign-up. Additionally, LEIP Deploy packages are now hosted on a public PYPI-compatible registry.
TensorRT 10 Support: Added support for NVIDIA TensorRT 10, enabling enhanced performance optimization for NVIDIA GPUs with the latest features and improvements.
Jetpack 6.2 Support: LEIP Deploy now supports NVIDIA Jetson devices with Jetpack 6.2. Support for Jetpack 5.1 and 6.0 has now been deprecated. Users running older versions of Jetpack should continue using LEIP Deploy 1.3 (leip-pylre<3.0).
Raspberry Pi 5 Support: LEIP Deploy now supports Raspberry Pi 5 running Debian 12.
API Improvements: Users can now run models more easily with a streamlined PyLRE interface that automatically selects backend, execution provider, and precision based on model format and optimization, removing the need to configure options manually.
Usage and Diagnostic Reporting: Depending on your license tier, LEIP Deploy may collect usage metrics and diagnostic reports to help improve product quality and performance.

Jetpack 6.0 Support: LEIP Deploy now supports NVIDIA Jetson devices with Jetpack 6.0.

Android Support: LEIP Deploy now supports Android with the introduction of the Android LRE. Android developers can integrate the Latent Runtime Engine into their applications and run optimized models directly on device. Consult the Android LRE API reference and Android LRE example application for more information.

TVM and ONNX Options at runtime: Users can access classes that provide configuration options for TVM and ONNX models. Included in these options is the ability to change model precision at runtime, depending on the precision at compilation.
64-bit ONNX Support: Users can now use 64-bit integer (int64) inputs in ONNX models.

Under certain conditions, running infer.py twice would throw an error on the second run. This issue has been fixed.
Replaced a missing __version__ attribute in the PyLRE package.

Runtime Framework is now LEIP Deploy.
Model Encryption: Extends model encryption to support ONNX formats.
New APIs: Added for retrieving model UUIDs and simplifying decryption of models at runtime, along with extending the capabilities for measuring model performance.