Skip to main content
Skip table of contents

LEIP Train

The leip train tool is used for automating the process of doing Quantization Guided Training (QGT) on a model. This is intended for first-time to moderate level users of the LEIP SDK. More advanced users might want details on using the python API for Quantization Guided Training directly. For information on the python API please see this document.

The  leip train tool accepts an input model and configuration JSON file, and generates a training script and automatically runs it. If you would prefer to edit the script before running it manually, you can supply the --invoke_training false option. Using the configuration JSON file you can specify the training dataset and control the training lambda parameters. The specifics about the fields are detailed in the sections below.

Please note that leip train currently supports Keras .h5 file format.

The leip train tool fits into the LEIP toolchain as shown in the following diagram (click to expand):

LEIP Toolchain

CLI Usage

The basic command is:

CODE
leip train --input_path path/to/model/ \
            --output_path output/ \
            --training_config_file path/to/config_file \
            --training_script_path destination_path_for_script \
            --invoke_training [true|false]

For a detailed explanation of each option see the CLI Reference for LEIP Train.

API Usage

Refer to the following for examples on Quantization Guided Training using the python API:

Config JSON File Fields

The training_config_file command line argument is optional. When it is supplied, it is a path to a JSON document containing any of the following fields. Default values will be used for any fields that are not present in the JSON document.

bits - (int) number of bits to quantize to - defaults to 8

batchsize - (int) batchsize used during training - defaults to 256

epochs - (int) number of epochs used during training - defaults to 10

epochs_pre_quantization - (int) the number of pre-quantization epochs to train on the model

quantizer - (string) quantization algorithm to use - defaults to “asymmetric”

dataset - (string or JSON) - please see the section below

regularizer_attachment_scheme - (JSON) - please see the section below

The Step Option

The --step option accept 3 possible values:

train - (default) this will execute the full training pipeline in the training script

list - this will cause the layer names and regularizers to be listed to stdout. this can be useful when creating the regularizer_attachment_scheme (see below)

dump_model_json - this will dump the entire model to the screen as JSON

Example Usage

The first step is to create and save the lenet model we are going to train with the mnist dataset. Add the following python code to a file named create_and_save_lenet_model.py.

PY
import os
import sys
import tensorflow as tf

def construct_model():
     tf.keras.backend.clear_session()
     model = tf.keras.Sequential(name="LeNet-5",)
     model.add(tf.keras.layers.InputLayer(input_shape=(28, 28)))
     model.add(tf.keras.layers.Reshape(target_shape=(28, 28, 1)))
     model.add(tf.keras.layers.Conv2D(filters=6, kernel_size=(3, 3), activation='relu',))
     model.add(tf.keras.layers.AveragePooling2D())
     model.add(tf.keras.layers.Conv2D(filters=16, kernel_size=(3, 3), activation='relu',))
     model.add(tf.keras.layers.AveragePooling2D())
     model.add(tf.keras.layers.Flatten())
     model.add(tf.keras.layers.Dense(units=120, activation='relu',))
     model.add(tf.keras.layers.Dense(units=84, activation='relu',))
     model.add(tf.keras.layers.Dense(units=10, activation='softmax',))
     model.summary()
     return model

def save_model(model, path):
     model.save(
         path,
         overwrite=True,
         include_optimizer=False,
         save_format="h5")

save_model(construct_model(), sys.argv[1]

Go ahead and run the script you just created, which will save the model to the lenet_model directory.

CODE
# create the destination directory
mkdir lenet_model

# run the script we created above
python3 create_and_save_lenet_model.py ./lenet_model/lenet.h5

Now, create a config file in your current directory and name it my_config.json

CODE
{
  "epochs_before_attach": 0,
  "epochs": 2,
  "bits": 5,
  "batchsize": 256
}

Next, using the leip zoo command download the mnist dataset to your local workspace

CODE
leip zoo download --dataset_id mnist --variant_id eval

Then run leip train pointing to the model we just created and the config file using the list step

CODE
leip train --input_path ./lenet_model/lenet.h5 \
           --training_config_file my_config.json \
           --output_path ./outputTrain \
           --step list

This will print out something similar to the following:

(the output has been truncated in this example)

CODE
INFO:root:{
    "conv2d_1": {
        "class_name": "Conv2D",
        "kernel_regularizer": null,
        "bias_regularizer": null,
        "kernel_constraint": null,
        "bias_constraint": null
    },
    "dense": {
        "class_name": "Dense",
        "kernel_regularizer": null,
        "bias_regularizer": null,
        "kernel_constraint": null,
        "bias_constraint": null
    }
    ...
}

Using the layer names displayed, we can then edit our my_config.json config JSON file to add the regularizer_attachment_scheme field with the appropriate . See the details on how this is done in the section below.

We can then run the training using the following command:

CODE
leip train --input_path ./lenet_model/lenet.h5 \
           --training_config_file my_config.json \
           --output_path ./outputTrain \
           --step train

Note the --step train is the default and doesn’t not need to be explicitly specified.

Finally, we could display the entire JSON representation of the trained model with:

CODE
leip train --input_path ./outputTrain/trained/leip_trained_model.h5 \
         --training_config_file my_config.json \
         --output_path ./outputTrain \
         --step dump_model_json

This will show the regularizers that were attached to the layers.

There are other steps we can take from here using the LEIP SDK. For information on how to compile the newly trained model see LEIP Compile .

The Dataset Field

The dataset field’s value can be of 2 types. When a string is supplied it is the name looked for in the tf.keras.datasets python module specifying the prepackaged dataset in the Tensorflow library.

CODE
"dataset": "mnist"

When the value is a JSON subdocument, it is used to specify a custom dataset supplied by the user and will have the following fields:

path - (string) a path to the index.txt file of the dataset. please see the the leip evaluate document for the format of the index.txt file

preprocessor - (string) (optional) the preprocessor that should be applied to the input

size - (int) (optional) amount of items in the dataset that should be used during training/validation

feeder - (string) (optional) which FileFeeder class to use

Here is an example dataset field with a custom dataset value:

CODE
"dataset": {
    "path": "mnist_dataset/index.txt",
    "preprocessor": None
    "size": 70000
}

An example case of when to override the feeder field is in the case of a detection model dataset that has both images and XML based annotations (such as the pascal-voc2007 dataset from the LEIP Model Zoo). You would add the field for the appropriate class such as: "feeder": "PascalVOCFeeder".

The Regularizer Attachment Scheme Field

This section describes the syntax for the regularizer_attachment_scheme field of the JSON config. It tells the Quantization Guided Training system how to attach regularizer JSON to the model description such that the regularizers guide the training towards the appropriate quantized weights.

The value of the fields is a JSON array, where each item in the array is a JSON subdocument with 2 fields: pattern and regularizers.

The pattern field is a simplified regular expression (regex) that matches a set of layer names in the model. As an example, if you would like to match any layer beginning with the prefix dense you would use dense*.

The regularizers field’s value is a JSON subdocument with 4 fields:

kernel_regularizer

bias_regularizer

kernel_constraint

bias_constraint

where the value of each of these fields is a JSON subdocument with implementation specific fields for that regularizer/constraint class.

To tie this all together, the following JSON snippet shows a complete example regularizer_attachment_scheme field with 2 entries, one with a pattern of dense* and one with a pattern of *. Note that for each layer in the model, the entries are checked for a match in order and the first matching entry will have its regularizers value attached to the layer. In this example we have made any layers whose name starts with dense quantize to 2 bits, and all other layers to 8 bits.

CODE
"regularizer_attachment_scheme": [
    {
        "pattern": "dense*",
        "regularizers": {
            "kernel_regularizer": {
                "class_name": "QuantizationGuidedRegularizer",
                "config": {
                    "num_bits": 2,
                    "lambda_1": 1.0,
                    "lambda_2": 1.0,
                    "lambda_3": 0.0,
                    "lambda_4": 1.0,
                    "lambda_5": 1.0,
                    "quantizer_name": "asymmetric"
                }
            },
            "bias_regularizer": {
                "class_name": "QuantizationGuidedRegularizer",
                "config": {
                    "num_bits": 2,
                    "lambda_1": 1.0,
                    "lambda_2": 1.0,
                    "lambda_3": 0.0,
                    "lambda_4": 1.0,
                    "lambda_5": 1.0,
                    "quantizer_name": "asymmetric"
                }
            },
            "kernel_constraint": null,
            "bias_constraint": null
        }
    },
    {
        "pattern": "*",
        "regularizers": {
            "kernel_regularizer": {
                "class_name": "QuantizationGuidedRegularizer",
                "config": {
                    "num_bits": 8,
                    "lambda_1": 1.0,
                    "lambda_2": 1.0,
                    "lambda_3": 0.0,
                    "lambda_4": 1.0,
                    "lambda_5": 1.0,
                    "quantizer_name": "asymmetric"
                }
            },
            "bias_regularizer": {
                "class_name": "QuantizationGuidedRegularizer",
                "config": {
                    "num_bits": 8,
                    "lambda_1": 1.0,
                    "lambda_2": 1.0,
                    "lambda_3": 0.0,
                    "lambda_4": 1.0,
                    "lambda_5": 1.0,
                    "quantizer_name": "asymmetric"
                }
            },
            "kernel_constraint": null,
            "bias_constraint": null
        }
    }
]

Please note that when this field is left blank, the training script will implicitly attach a regularization scheme to all quantizable layers which is the equivalent of the following:

CODE
"regularizer_attachment_scheme": [
    {
        "pattern": "*",
        "regularizers": {
            "kernel_regularizer": {
                "class_name": "QuantizationGuidedRegularizer",
                "config": {
                    "num_bits": 8,
                    "lambda_1": 1.0,
                    "lambda_2": 1.0,
                    "lambda_3": 0.0,
                    "lambda_4": 1.0,
                    "lambda_5": 1.0,
                    "quantizer_name": "asymmetric"
                }
            },
            "bias_regularizer": {
                "class_name": "QuantizationGuidedRegularizer",
                "config": {
                    "num_bits": 8,
                    "lambda_1": 1.0,
                    "lambda_2": 1.0,
                    "lambda_3": 0.0,
                    "lambda_4": 1.0,
                    "lambda_5": 1.0,
                    "quantizer_name": "asymmetric"
                }
            },
            "kernel_constraint": null,
            "bias_constraint": null
        }
    }
]

In addition to accepting a JSON array, the regularizer_attachment_scheme can accept a string which is interpreted as a separate file where the JSON array is stored. This allows one to keep the actual mappings of layer name regular expressions to regularizer JSON documents in separate files for easier A/B testing.


JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.