Milk-V Duo Development Board in Action - Image Classification with MobileNetV2

CV1800B(Duo English Forum)
1

Milk-V Duo Development Board Practical Guide – Image Classification Based on MobileNetV2

This tutorial introduces the conversion of the MobileNet-Caffe model using the TPU-MLIR toolchain, generating MLIR and quantizing it into an INT8 model. It also covers deployment and testing on the Milk-V Duo development board for image classification tasks. The following steps are involved:

:triangular_flag_on_post:【Note】:triangular_flag_on_post:: The Milk-V Duo development board is equipped with the CV1800B chip, which supports ONNX series and Caffe models. Currently, it does not support TFLite model quantization data types but supports BF16 format quantization and INT8 format asymmetric quantization.

  1. Environment Setup
  2. MobileNet-Caffe Model Conversion
  3. Deploying INT8 cvimodel to the Duo Development Board and Verification

Let’s dive into each of these steps in detail.

1. Environment Setup

1.1 Configure Docker Development Environment

Install and configure Docker:

sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker

Pull the Docker image from Docker Hub:

docker pull sophgo/tpuc_dev:v2.2

Run Docker to create a container with the name ‘duodev’ (you can modify it) and set the default directory to ‘/workspace’:

docker run --privileged --network=host --name duodev -v $PWD:/workspace -it sophgo/tpuc_dev:v2.2

Inside the Docker environment, configure the network and install basic dependencies:

apt-get update
apt-get install net-tools

Download the tpu-mlir model conversion toolchain. The package naming format is tpu-mlir_xxxx.tar.gz, where xxxx is the version number. In this tutorial, we use version v1.2.89-g77a2268f-20230703 as an example:

wget --user='cvitek_mlir_2023' --password='7&2Wd%cu5k' ftp://218.17.249.213/home/tpu-mlir_v1.2.89-g77a2268f-20230703.tar.gz

Unpack the toolchain and set environment variables:

tar zxf tpu-mlir_v1.2.89-g77a2268f-20230703.tar.gz
source tpu-mlir_v1.2.89-g77a2268f-20230703/envsetup.sh

1.2 Prepare Working Directory

Download the MobileNet model from the official website:

git clone https://github.com/shicai/MobileNet-Caffe.git

Create a directory named mobilenet_v2 and copy the model files from the cloned MobileNet-Caffe directory (referred to as ${MOBILE_DIR}), as well as the image files from the tpu-mlir toolchain directory (referred to as ${TPUMLIR_DIR}). Create a working directory named workspace (referred to as ${WORK_DIR}) to store the compiled MLIR and cvimodel files:

mkdir mobilenet_v2 && cd mobilenet_v2
cp ${MOBILE_DIR}/mobilenet_v2_deploy.prototxt .
cp ${MOBILE_DIR}/mobilenet_v2.caffemodel .
cp -rf ${TPUMLIR_DIR}/regression/dataset/ILSVRC2012/ .
cp -rf ${TPUMLIR_DIR}/regression/image/ .
mkdir workspace && cd workspace

image
image1097×167 64.5 KB

2. MobileNet-Caffe Model Conversion

The model conversion steps are as follows:

  1. Convert the Caffe model to MLIR.
  2. Generate calibration tables required for quantization.
  3. Quantize MLIR into INT8 asymmetric cvimodel.

2.1 Convert Caffe Model to MLIR

Before converting the model, it’s important to understand the model’s preprocessing. If the model uses a preprocessed npz file as input, you don’t need to worry about preprocessing. The preprocessing can be expressed by the following formula (x represents the input):

image

In this example, the model has BGR input, with mean and scale values of 103.94,116.78,123.68 and 0.017,0.017,0.017, respectively. Here’s the model conversion command:

model_transform.py \
    --model_name mobilenet_v2 \
    --model_def ../mobilenet_v2_deploy.prototxt \
    --model_data ../mobilenet_v2.caffemodel \
    --input_shapes [[1,3,224,224]] \
    --resize_dims=256,256 \
    --mean 103.94,116.78,123.68 \
    --scale 0.017,0.017,0.017 \
    --pixel_format bgr \
    --test_input ../image/cat.jpg \
    --test_result mobilenet_v2_top_outputs.npz \
    --mlir mobilenet_v2.mlir

The files generated by executing the model_transform.py script are shown in the following image:

image
image1401×108 60 KB

2.2 Generate Calibration Tables

Run run_calibration.py to generate calibration tables. Prepare around 100 to 1000 input data samples, depending on your situation. Here, we use 100 images from ILSVRC2012 as an example:

run_calibration.py mobilenet_v2.mlir \
    --dataset ../ILSVRC2012 \
    --input_num 100 \
    -o mobilenet_v2_cali_table

After running this, a file named ${model_name}_cali_table will be generated. This file is used for input to compile the INT8 model.

image
image1405×170 79.8 KB

2.3 Quantize MLIR into INT8 Asymmetric cvimodel

:triangular_flag_on_post:【Note】:triangular_flag_on_post:: The Milk-V Duo development board supports ONNX series and Caffe models, and it currently does not support TFLite model quantization data types. It supports BF16 format quantization and INT8 format asymmetric quantization. Therefore, in this section, the model_deploy.py script uses the asymmetric option for asymmetric quantization.

To convert the MLIR file into an INT8 asymmetric quantized model, execute the following command:

model_deploy.py \
    --mlir mobilenet_v2.mlir \
    --asymmetric \
    --calibration_table mobilenet_v2_cali_table \
    --fuse_preprocess \
    --customization_format BGR_PLANAR \
    --chip cv180x \
    --quantize INT8 \
    --test_input ../image/cat.jpg \
    --model mobilenet_v2_cv1800_int8_asym.cvimodel

After compilation, a file named ${model_name}_cv1800_int8_asym.cvimodel will be generated, as shown in the following image:

image
image1407×236 120 KB

3. Deploy INT8 cvimodel to the Duo Development Board and Verify

This document does not elaborate on the configuration of the Duo’s working environment. It is assumed that a successful connection to the development board has already been established. The connection information for the Duo development board is as follows:
IP: 192.168.42.1
User: root
Password: milkv

Download the cvitek_tpu_sdk required to run on the development board:

wget --user='cvitek_mlir_2023' --password='7&2Wd%cu5k' ftp://218.17.249.213/home/tpu_sdk_t4.1.0-14-g3e77050/cvitek_tpu_sdk_cv180x_musl_riscv64_rvv.tar.gz

Upload the cvitek_tpu_sdk package to the Duo development board:

scp cvitek_tpu_sdk_cv180x_musl_riscv64_rvv.tar.gz root@192.168.42.1:/cvitek_tpu_sdk_cv180x_musl_riscv64_rvv.tar.gz

In the ${WORK_DIR} directory, copy the generated ${model_name}_cv1800_int8_asym.cvimodel to the Duo development board:

scp ${model_name}_cv1800_int8_asym.cvimodel root@192.168.42.1:/${model_name}_cv1800_int8_asym.cvimodel

Note: The following content in this section should be performed on the Duo development board.

SSH into the Duo development board, and you can see the cvitek_tpu_sdk package and cvimodel that were transferred earlier:

ssh root@192.168.42.1

image
image1476×162 19.1 KB

Unpack cvitek_tpu_sdk_cv180x_musl_riscv64_rvv.tar.gz, set environment variables, and navigate to the samples directory for testing:

# Unpack the package
tar zxf cvitek_tpu_sdk_cv180x_musl_riscv64_rvv.tar.gz

# Set the TPU_ROOT to the directory of cvitek_tpu_sdk, in this example, TPU_ROOT=/cvitek_tpu_sdk
export TPU_ROOT=$PWD/cvitek_tpu_sdk

# Navigate to the SDK directory and set environment variables
cd cvitek_tpu_sdk && source ./envs_tpu_sdk.sh

# Print cvimodel info, $MODEL_PATH is the directory where the cvimodel is located
cd samples
./bin/cvi_sample_model_info $MODEL_PATH/${model_name}_cv1800_int8_asym.cvimodel

# Testing
./bin/cvi_sample_classifier_fused_preprocess \
    $MODEL_PATH/${model_name}_cv1800_int8_asym.cvimodel \
    ./data/cat.jpg \
    ./data/synset_words.txt

image
image851×367 93.6 KB

Note:

  1. The samples directory contains samples_extra, which provides more sample scripts. However, the cvimodel name is hardcoded in these scripts. If you want to use the scripts, you will need to modify the cvimodel name manually.
  2. This section describes deploying and testing the precompiled sample programs on the converted cvimodel. If developers are interested in coding and cross-compiling the sample source code themselves, please refer to Chapter 9 in the official TPU-MLIR documentation under the Section 3 “Compile and Run Runtime Sample.”

Appendix

The summary of files referenced in the main text is as follows:

  • TPU-MLIR Model Conversion Toolchain: tpu-mlir_v1.2.89-g77a2268f-20230703.tar.gz
  • TPU SDK Development Toolkit: cvitek_tpu_sdk_cv180x_musl_riscv64_rvv.tar.gz
  • (Appendix) Sample Test Program Source Code: cvitek_tpu_samples.tar.gz
  • (Appendix) Pre-converted cvimodel Package: cvimodel_samples_cv180x.tar.gz

The files required for TPU development mentioned in the main text can be obtained from the following SFTP site:

sftp://218.17.249.213
User: cvitek_mlir_2023
Password: 7&2Wd%cu5k

image
image1446×471 192 KB

Alternatively, you can directly obtain them using wget:

# TPU-MLIR Model Conversion Toolchain
wget --user='cvitek_mlir_2023' --password='7&2Wd%cu5k' ftp://218.17.249.213/home/tpu-mlir_v1.2.89-g77a2268f-20230703.tar.gz

# TPU SDK Development Toolkit
wget --user='cvitek_mlir_2023' --password='7&2Wd%cu5k' ftp://218.17.249.213/home/tpu_sdk_t4.1.0-14-g3e77050/cvitek_tpu_sdk_cv180x_musl_riscv64_rvv.tar.gz

# (Appendix) Sample Test Program Source Code
wget --user='cvitek_mlir_2023' --password='7&2Wd%cu5k' ftp://218.17.249.213/home/tpu_sdk_t4.1.0-14-g3e77050/cvitek_tpu_samples.tar.gz

# (Appendix) Pre-converted cvimodel Package
wget --user='cvitek_mlir_2023' --password='7&2Wd%cu5k' ftp://218.17.249.213/home/tpu_sdk_t4.1.0-14-g3e77050/cvimodel_samples_cv180x.tar.gz