Model compilation

Goal

In this tutorial you will convert the quantized model into a format that’s compatible with the target DPU platform and can leverage the hardware acceleration on the edge.

Prerequisites

  1. Docker installed.

  2. Model repository and environment set up as described in Setup the project environment .

  3. Quantized model created as described in Model quantization.

  4. Target platform fingerprint arch.json file generated by Vivado as described in Leopard or Antelope tutorials.

Provided outputs

Following (Tutorial files) files are associated with this tutorial:

  • ML deployment/03 Model compilation/deep_globe_segmentation_unet_512_512.xmodel - compiled model

  • Leopard/Zero-to-hero/04 Deep learning Processor Unit/arch.json - DPU fingerprint file for Leopard

  • Antelope/Zero-to-hero/04 Deep learning Processor Unit/arch.json - DPU fingerprint file for Antelope

Prepare for compilation Machine learning workstation

  1. Make sure that arch.json produced by Vivado resides in ~/sml-tutorials/ml-deployment/arch.json.

  2. Enter the Vitis AI deployment container with the working directory volume mounted:

    customer@ml-workstation:~/sml-tutorials/ml-deployment$ docker run \
    -it \
    -v "$(pwd)":/workspace \
    -e UID="$(id -u)" -e GID="$(id -g)" \
    xilinx/vitis-ai-pytorch-cpu:ubuntu2004-3.5.0.306

Model compilation Vitis AI deployment container

Run the following commands in the container environment.

  1. Activate the desired conda environment for PyTorch models deployment:

    vitis-ai-user@vitis-ai-container-id:/workspace$ conda activate vitis-ai-wego-torch2

  2. Run the compiler command on the quantized model to produce the FPGA-acceleration-compatible model based on the provided arch.json:

    (vitis-ai-wego-torch2) vitis-ai-user@vitis-ai-container-id:/workspace$ vai_c_xir \
     --xmodel ./output/03-quantize/Unet_int.xmodel \
     --arch ./arch.json \
     --output_dir ./output/04-compile \
     --net_name deep_globe_segmentation_unet_512_512

    The compiled model should appear in the ~/sml-tutorials/ml-deployment/04-compile directory. If you wish to skip this step it’s also available via git-lfs. Mind that the model is compiled spcecifically for the DPU architecture identified by the arch.json file.

    Note

    Mind that compilation may strip the model of the layers unsupported by the target platform. The compiler will tell you about the removed layers if there are any. These layers must be re-implemented from scratch on the target platform on the CPU side.

  1. Exit the Vitis AI container: exit.