Detector BYOD: Pascal VOC Format Example

Bring Your Own Pascal VOC Formatted Data

Training with a Pascal VOC formatted dataset is very similar to the using COCO formatted data. Perform the following to train your model with Pascal VOC formatted data:

Find the Pascal-like dataset template file in the Docker container located at

/latentai/custom-dataset-configs/data/pascal-reader-template.yaml

It will look like this:

CODE

defaults:
  - /data/transforms@module.dataset_generator.train_transforms: resize.yaml
  - /data/transforms@module.dataset_generator.valid_transforms: resize.yaml
  - _self_

nclasses: ???
module:
  _target_: af.core.data.modules.adaptermodule.AdaptorDataModule
  batch_sizes: ${task.batch_sizes}
  num_workers: ${task.num_workers}
  adaptors: ${model.adaptors}
  dataset_generator:
    _target_: af.core.data.generic.pascal_reader.PascalReader
    root_path: null
    images_dir: JPEGImages
    annotations_dir: Annotations
    type: detection
    is_split: false
    trainval_split_ratio: 0.75
    trainval_split_seed: 42
    train_set: ImageSets/Main/train.txt
    val_set: ImageSets/Main/val.txt
    labelmap_file: pascal_label_map.json
    dataset_name: my-custom-pascal-like-data

The fields you will need to change are for the number of classes in your dataset, and the Reader settings shown here from line 10 on. Changing those fields with your dataset’s information is enough to load and use your PascalVOC-formatted dataset.

nclasses: The number of classes in your dataset.
root_path: The absolute path to the root directory of the dataset.
images_dir: The path to folder containing only images, relative to root_path.
annotations_dir: The path to folder containing xml files, relative to root_path.
type: detection or segmentation. We can leave the value as detection for this detection recipe.
is_split: true or false.
- If set to true, text files containing the list of samples for training and validation should be specified using train_set and val_set
- If set to false, data will be split by the ingestor given the trainval_split_ratio and trainval_split_seed
trainval_split_ratio: The ratio to use to split the dataset. Used only if is_split: false
trainval_split_seed: The seed to use to pseudo randomly split the dataset. Used only if is_split: false
train_set: The path to text file containing names (no extensions) to the training samples. Used only if is_split: true
val_set: The path to text file containing names (no extensions) to the validation samples. Used only if is_split: true
labelmap_file: (Optional) The path to a json file containing a map from class index (int) to class name (string), relative to root_path. If you don't have this file, it will be created automatically on the first run, and stored to streamline future runs.
dataset_name: This string will be used to name any generated artifacts.

A Concrete BYOD Pascal VOC Format Example

Smoke Detection Dataset Overview

Download and extract the dataset.

CODE

mkdir /latentai/workspace/datasets
cd /latentai/workspace/datasets
wget https://s3.us-west-1.amazonaws.com/leip-showcase.latentai.io/recipes/smoke_dataset.zip
unzip smoke_dataset.zip

# Verify data location matches the directories in this example:
ls /latentai/workspace/datasets/smoke_dataset
Annotations  ImageSets  JPEGImages

# Get back to /latentai
cd /latentai

Using the Template to Ingest the Smoke Detection Data

Now we need to create a YAML configuration file for this dataset. Create a new file custom-configs/data/smoke_config.yaml by copying our template:

CODE

cp /latentai/custom-configs/data/pascal-reader-template.yaml \
 /latentai/custom-configs/data/smoke_config.yaml

We will now edit the template with the correct values for the dataset_generator section and the correct number of classes. After filling out your configuration, custom-configs/data/smoke_config.yaml should look like this:

CODE

defaults:
  - /data/transforms@module.dataset_generator.train_transforms: resize.yaml
  - /data/transforms@module.dataset_generator.valid_transforms: resize.yaml
  - _self_

nclasses: 1
module:
  _target_: af.core.data.modules.adaptermodule.AdaptorDataModule
  batch_sizes: ${task.batch_sizes}
  num_workers: ${task.num_workers}
  adaptors: ${model.adaptors}
  dataset_generator:
    _target_: af.core.data.generic.pascal_reader.PascalReader
    root_path: /latentai/workspace/datasets/smoke_dataset
    images_dir: JPEGImages
    annotations_dir: Annotations
    type: detection
    is_split: true
    trainval_split_ratio: null
    trainval_split_seed: null
    train_set: ImageSets/Main/train.txt
    val_set: ImageSets/Main/val.txt
    labelmap_file: pascal_label_map.json
    dataset_name: smoke-pascal-like

Use command=vizdata to see what a few samples of the data will look like:

CODE

af data=smoke_config command=vizdata

The vizdata command will read the annotations and draw bounding boxes over some of the samples. The sample images will be stored in /latentai/artifacts/vizdata/smoke/*.

Train

You can train a model by selecting any of our detector recipes and using command=train:

CODE

af --config-name=<recipe_name> data=smoke_config \
 command=train task.moniker="BYOD_recipe"

Note where the checkpoint is stored at the end of the training run. The checkpoint will be stored in a path of the form:/latentai/artifacts/train/{date}_{time}_BYOD_recipe/{epoch}.ckpt Find that file and store it for ease of use in the next steps:

CODE

export SMOKE_CHECK=<path to .ckpt file>

# Example:
# export SMOKE_CHECK=/latentai/artifacts/train/2022-08-23_17-25-29_task_BYOD_recipe/epoch-19_step-1840.ckpt

Evaluate on the Host Before Exporting

You can now use your newly trained model by passing the checkpoint to the commands you used previously with the pre-trained model. Perform the following to evaluate.

CODE

af --config-name=<recipe_name> data=smoke_config \
  command=evaluate task.moniker="BYOD_recipe" \
  +checkpoint=$SMOKE_CHECK

Export the Model

CODE

af --config-name=<recipe_name> data=smoke_config \
  command=export task.moniker="BYOD_recipe" \
  +export.include_preprocessors \
  +checkpoint=$SMOKE_CHECK

Your exported model will be saved in:
/latentai/artifacts/export/<some_name>/traced_model.pt

Save the traced_model.pt path as a variable:

CODE

export EXPORTED_MODEL=/latentai/artifacts/export/<some_name>/traced_model.pt

Optimize the Model

You will need to identify a calibration dataset from your BYOD data before you can optimize your trained model. Identifying an ideal calibration dataset is beyond the scope of this tutorial, but it usually entails selecting a few representative images for the quantizer. If you are targeting a CUDA device, you will need to provide 2N images, where N is your batch size. You will need to provide a path to these images in the file rep_dataset.txt that is in the model recipe directory. We will use the last two images in the validation dataset for the Smoke dataset.

Override the file in /latentai/recipes/<recipe_name>/rep_dataset.txt with paths to samples in the Smoke dataset:

CODE

/latentai/workspace/datasets/smoke_dataset/JPEGImages/ck0ukkz8tytsn0721y70sud46_jpeg.rf.18cbd5352f166266b09e38e04afa7394.jpg
/latentai/workspace/datasets/smoke_dataset/JPEGImages/ck0uklx72wly70701px5wl6fw_jpeg.rf.cb1683418c777e1d072763d02d84465b.jpg

Try using different calibration images if you find that your optimized (INT8) model has substantial accuracy loss from the compiled (Float32) model. Some image sets will produce outlier results that may adversely affect accuracy.

Please bear in mind that the optimal training dataset may change each time you retrain the model. The above is an example dataset, but you may need to test out different datasets to yield the best results after you have trained your model.

Once you have exported your trained model and determined a calibration dataset, simply compile and optimize it as before, providing the traced model as the input path:

CODE

leip pipeline \
  --input_path $EXPORTED_MODEL \
  --output_path /latentai/workspace/output/byod_smoke \
  --config_path /latentai/recipes/<recipe_name>/pipeline_x86_64_cuda.yaml

Evaluate the Smoke Detection Example on Host

You will need to convert the Pascal VOC formatted data to COCO format if you would like to use leip evaluate to evaluate your model. You can do this via the af command=export_data function:

CODE

# Export the Pascal data into COCO format:
af data=smoke_config command=export_data

# Some processes will look for the instances file in the data directory:
cp /latentai/artifacts/export_data/COCO/smoke/instances_val2017.json \
  /latentai/artifacts/export_data/COCO/smoke/val2017/

# Data will be available here in COCO format:
# /latentai/artifacts/export_data/COCO/smoke

You can now evaluate your compiled, optimized model against the Smoke dataset:

CODE

# Evaluate Float32:
leip evaluate \
  --input_path /latentai/workspace/output/byod_smoke/Float32-compile/ \
  --test_path /latentai/artifacts/export_data/COCO/smoke/val2017/instances_val2017.json \
  --dataset_type coco

# Evaluate Int8:
leip evaluate \
  --input_path /latentai/workspace/output/byod_smoke/Int8-optimize/ \
  --test_path /latentai/artifacts/export_data/COCO/smoke/val2017/instances_val2017.json \
  --dataset_type coco

Evaluate the Smoke Detection Example on Target

You will run leip evaluate in the SDK Docker container with inference performed on the device under test in order to evaluate on a remote target device. You will first need to setup your target by installing Latent AI Object Runner (LOR). You will then evaluate using the LRE objects created by leip pipeline.

CODE

# Substitute the IP address of your target device for <IP_ADDR>
# The default port for LOR is 50051

# Evaluating Float32:
leip evaluate \
  --input_path /latentai/workspace/output/byod_smoke/Float32-package \
  --host <IP_ADDR> --port 50051 \
  --test_path /latentai/artifacts/export_data/COCO/smoke/val2017/instances_val2017.json \
  --dataset_type coco

# Evaluating Int8:
leip evaluate \
  --input_path /latentai/workspace/output/byod_smoke/Int8-package \
  --host <IP_ADDR> --port 50051 \
  --test_path /latentai/artifacts/export_data/COCO/smoke/val2017/instances_val2017.json \
  --dataset_type coco