2.1. Training for Basic Classification

This tutorial covers training a neural network model on a GPU server to classify images of 10 objects in the CIFAR-10 dataset.

2.1.1. Preparation

The CIFAR-10 dataset is available from the official website. However, the dataset’s original format (numpy array) is unsupported in Blueoil.

Blueoil supports 2 formats for basic classification:

  • Caltech101 format

  • DeLTA-Mark format

Note: Please see the details in Prepare training dataset

You can download PNG format CIFAR-10 from our mirror site. https://s3-ap-northeast-1.amazonaws.com/leapmind-public-storage/datasets/cifar.tgz

$ wget https://s3-ap-northeast-1.amazonaws.com/leapmind-public-storage/datasets/cifar.tgz
$ tar xzf cifar.tgz

The subdirectories of train and test represent the classes. The respective images are located in the aforementioned subdirectories. (The organization is the same for the Caltech101 format.)

cifar
 ├─ train
 │   ├─ airplane
 │   │    ├─ xxxx_airplane.png
 │   │    ├─ yyyy_airplane.png
 │   │    ...
 │   ├─ automobile
 │   │    ├─ ...
 │   ├─ bird
 │   │    ├─ ...
 │   ├─ ...
 │   └─ truck
 │        ├─ ...
 └─ test
     ├─ airplane
     │    ├─ ...
     ├─ ...
     ...

The CIFAR-10 dataset consists of 60,000 32x32 color images split into 10 classes, with 6,000 images per class. There are 50,000 training images and 10,000 test images.

2.1.2. Generate a configuration file

Generate your model configuration file interactively by running the blueoil init command.

$ docker run --rm -it \
    -v $(pwd)/config:/home/blueoil/config \
    blueoil_$(id -un):{TAG} \
    blueoil init -o config/my_config.yml

The {TAG} value must be set to a value like v0.15.0-15-gf493ec9 that can be obtained with the docker images command. This value depends on your environment.

Below is an example configuration.

#### Generate config ####
  your model name ():  cifar10_test
  choose task type:  classification
  choose network:  LmnetV1Quantize
  choose dataset format:  Caltech101
  training dataset path:  /home/blueoil/cifar/train/
  set validataion dataset? (if answer no, the dataset will be separated for training and validation by 9:1 ratio.)  yes
  validataion dataset path:  /home/blueoil/cifar/test/
  batch size (integer):  64
  image size (integer x integer):  32x32
  how many epochs do you run training (integer):  100

  • Model name: (Any)

  • Task type: classification

  • Network: LmnetV1Quantize / LmnetV0Quantize

  • Dataset format: Caltech101

  • Dataset path:

    • Training: {dataset_dir on docker container}/cifar/train/

    • Validation: {dataset_dir on docker container}/cifar/test/

  • Batch size: (Any)

  • Image size: 32x32

  • Number of epoch: (Any number)

If configuration finishes, the configuration file is generated in the my_config.yml under config directory.

2.1.3. Train a neural network

Train your model by running blueoil train with model configuration.

$ docker run --rm \
    -e CUDA_VISIBLE_DEVICES=0 \
    -v $(pwd)/cifar:/home/blueoil/cifar \
    -v $(pwd)/config:/home/blueoil/config \
    -v $(pwd)/saved:/home/blueoil/saved \
    blueoil_$(id -un):{TAG} \
    blueoil train -c config/my_config.yml

Just like init, set the value of {TAG} to the value obtained by docker images. Change the value of CUDA_VISIBLE_DEVICES according to your environment.

When training has started, the training log and checkpoints are generated under ./saved/{MODEL_NAME}. The value of {MODEL_NAME} will be train_{TIMESTAMP}.

Training is running on TensorFlow backend. So you can use TensorBoard to visualize your training process.

$ docker run --rm \
    -p 6006:6006 \
    -v $(pwd)/saved:/home/blueoil/saved \
    blueoil_$(id -un):{TAG} \
    tensorboard --logdir=saved/{MODEL_NAME}

  • Loss / Cross Entropy, Loss, Weight Decay

  • Metrics / Accuracy

2.1.4. Convert training result to FPGA ready format.

Convert trained model to executable binary files for x86, ARM, and FPGA. Currently, conversion for FPGA only supports Intel Cyclone® V SoC FPGA.

$ docker run --rm \
    -e CUDA_VISIBLE_DEVICES=0 \
    -e OUTPUT_DIR=/home/blueoil/saved \
    -v $(pwd)/saved:/home/blueoil/saved \
    blueoil_$(id -un):{TAG} \
    blueoil convert -e {MODEL_NAME}

blueoil convert automatically executes some conversion processes.

  • Converts Tensorflow checkpoint to protocol buffer graph.

  • Optimizes graph.

  • Generates source code for executable binary.

  • Compiles for x86, ARM and FPGA.

If conversion is successful, output files are generated under ./saved/train_{TIMESTAMP}/export/save.ckpt-{Checkpoint No.}/{Image size}/output.

output
 ├── fpga (include preloader and FPGA configuration file)
 │   ├── preloader-mkpimage.bin
 │   ├── soc_system.rbf
 │   └── soc_system.dtb
 ├── models
 │   ├── lib (include trained model library)
 │   │   ├── lib_arm.so
 │   │   ├── lib_fpga.so
 │   │   └── lib_x86.so
 │   └── meta.yaml (model configuration)
 ├── python
 │   ├── lmnet (include pre-process/post-process)
 │   ├── README.md
 │   ├── requirements.txt
 │   ├── run.py (inference script)
 │   └── usb_camera_demo.py (demo script for object detection and classification)
 └── README.md

2.1.5. Run inference script on x86 Linux (Ubuntu 16.04)

  • Prepare inference images (not included in the training dataset)

    [ for example: airplane ]

  • Run inference script

    Explore into the output/python directory, and run run.py. Inference result is saved in ./output/output.json.

    Note: If you run the script for the first time, you have to setup a python environment (2.7 or 3.5+) and install the required python packages.

    $ cd {output/python directory}
$ sudo pip install -r requirements.txt  # only the first time
$ python run.py \
    -i {inference image path} \
    -m ../models/lib/lib_x86.so \
    -c ../models/meta.yaml

  • Check inference result. Should look like the example below.

    {
    "classes": [
        { "id": 0, "name": "airplane" },
        { "id": 1, "name": "automobile" },
        ...
        { "id": 9, "name": "truck" }
    ],
    "date": "yyyy-mm-ddThh:mm:ss.nnnnnnn",
    "results": [
        {
            "file_path": "{inference image path}",
            "prediction": [
                {
                    "class": { "id": 0, "name": "airplane" },
                    "probability": "0.95808434"
                },
                {
                    "class": { "id": 1, "name": "automobile" },
                    "probability": "0.0007377896"
                },
                ...
                {
                    "class": { "id": 9, "name": "truck" },
                    "probability": "0.008015169"
                }
            ]
        }
    ],
    "task": "IMAGE.CLASSIFICATION",
    "version": 0.2
}