Skip to content

Docker Installation

CUDA 12.1 Bug

Due to a known bug with CUDA versions greater than 12.1, executing compiled models on Orin Jetpack 5.X may fail.

Cross Compilation Support

Cross compilation allows users to compile for a aarch64 device (say Jetson or Raspberry Pi) on an x86_64 host device (say Intel or AMD CPU). If you wish to compile for aarch64 you will need to install g++-aarch64-linux-gnu. You can use APT to install this on Ubuntu systems: 

sudo apt-get install -y g++-aarch64-linux-gnu

1. Access Latent AI's Container Repository

In order to pull containers or install packages from Latent AI, you'll need to create a personal access token. To do so, follow these steps:

i. Login to the Latent AI Repository

  • Click the Sign in link in the upper right.
  • Select Sign In with SSO.
  • Enter your access credentials.

ii. Create your Personal Access Token

  • Click on your profile in the upper right.
  • Select User Token on the left navigation.
  • Select the Access user token button.
  • View your user token name and user token pass code.

You can export the token name and token pass code as environment variables:

export REPOSITORY_TOKEN_NAME=<user_token_name>
export REPOSITORY_TOKEN_PASSCODE=<user_token_pass_code>

2. Export License Key

You should have received this via email.

Tip

You can persist variables like LICENSE_KEY across bash sessions by placing them in your ~/.bashrc file.

export LICENSE_KEY=key/<license key>

3. Access the Latent AI Repository

docker login repository.latentai.com -u $REPOSITORY_TOKEN_NAME -p $REPOSITORY_TOKEN_PASSCODE

4. Start the Docker Container

The following command will pull the most recent leip-design image and start a Docker container with GPU support, shared file access, and a Jupyter Notebook interface accessible from your web browser at http://localhost:8888.

docker run Flag Explanations
  • -e LICENSE_KEY=$LICENSE_KEY: This option sets the environment variable LICENSE_KEY inside the container. The LICENSE_KEY is required by LEIP Design to validate your license. This key was exported to your environment in Step Two above.
  • -p $PORT:8888: This option maps the port of your choice on the host machine to port 8888 inside the container. Jupyter Notebook (which is launched inside the container) listens on port 8888 by default. This mapping allows you to access the Jupyter Notebook from your local machine through http://localhost:$PORT.
  • -p 6006:6006: This option maps port 6006 on the host machine to port 6006 inside the container for use by TensorBoard.
  • --gpus all: This flag tells Docker to make all available GPUs on the host system accessible inside the container. It enables the use of GPU acceleration for deep learning models, which is essential for many tasks within LEIP Design. If your system doesn't have a GPU or CUDA support, you can omit this flag.
  • --shm-size=12g: Sets the shared memory size for the container to 12 GB. This value cannot exceed the available memory of your host machine; if necessary, reduce the value appropriately.
  • -v $DIR:/latentai: The -v flag mounts a directory from your host machine to a specific directory inside the container (/latentai). (If the host directory doesn't exist, it will be created.) This allows you to share files between your host system and the container. Any changes made to files in this directory will be reflected both inside and outside the container.
  • repository.latentai.com/leip-design: This specifies the Docker image to use. It pulls the leip-design image from the Latent AI Docker repository and runs it. You need access credentials (as described earlier in the documentation) to pull this image.

To get started, we'll set some variables:

export PORT=<port to map for Jupyter access>

export DIR=<local directory to mount>

export NAME=<name to assign container>

docker run \
    --name $NAME -e LICENSE_KEY=$LICENSE_KEY -e TZ=America/New_York \
    -p $PORT:8888 -p 6006:6006 \
    --gpus all --shm-size=12g \
    -v $DIR:/latentai \
    repository.latentai.com/leip-design