Quick start: Run CNN models on RK3576

This guide follows NPU driver check → Runtime and Toolkit Lite2 installation → PC model conversion → C++ build and deployment → on-board inference. It uses YOLO11n from RKNN Model Zoo on the RK3576 built-in NPU, using SSH/SCP or a local board terminal throughout.

#Before you begin

Where to run commands: “On PC” means the Ubuntu development computer; “On board” means an SSH session or local board terminal. BOARD_IP is a placeholder: replace it with your board's LAN IP address.

If SSH is not enabled on the board, run this locally on the board first:

sudo apt update
sudo apt install openssh-server
sudo systemctl enable --now ssh
hostname -I

Then connect from the PC. On first connection, verify the host fingerprint and enter your account password when prompted:

ssh rk3576@BOARD_IP

#1. Fixing a missing NPU driver

#1.1 Kernel driver versus user-space software

The RK3576 NPU driver is RKNPU in the kernel, controlled by CONFIG_ROCKCHIP_RKNPU. Neither the Runtime .so nor the Toolkit Lite2 Python wheel can replace the kernel driver.

#1.2 Check the driver, device node, and load

On board:

uname -r
sudo dmesg | grep -i rknpu
sudo cat /sys/kernel/debug/rknpu/version
sudo cat /sys/kernel/debug/rknpu/load
ls -l /sys/class/drm/renderD*/device/driver

Excerpt from the actual test output:

RKNPU driver: v0.9.8
NPU load:  Core0:  0%, Core1:  0%,

/sys/class/drm/renderD129/device/driver -> .../bus/platform/drivers/RKNPU

renderD129 was the NPU node in this test, not a fixed device number. Identify it by the device/driver link to RKNPU. Other render nodes may belong to the display controller or GPU; /dev/dri alone does not prove the NPU is working.

If debugfs is not mounted, run:

mountpoint -q /sys/kernel/debug || sudo mount -t debugfs debugfs /sys/kernel/debug

Retry the version check. A missing debugfs query file alone does not prove the driver is absent; also check boot logs and the driver binding.

Check the kernel configuration on the board:

grep '^CONFIG_ROCKCHIP_RKNPU=' /boot/config-$(uname -r)

Actual output:

CONFIG_ROCKCHIP_RKNPU=y

=y means the driver is built into the kernel, so rknpu being absent from lsmod does not mean the driver is missing. The official RKNN SDK V2.3.2 quick-start guide recommends RKNPU driver 0.9.2 or later; inference was verified here with 0.9.8.

#1.3 If the driver is missing or needs updating

This guide uses the Seeed/Armbian kernel for reComputer RK3576. If the driver checks pass, go straight to Section 2; do not reinstall the kernel.

On the same board model with the Seeed package source configured, first inspect the kernel and device-tree packages:

sudo apt update
apt-cache policy linux-image-vendor-seeed-rk3576 linux-dtb-vendor-seeed-rk3576

After confirming both packages come from the board's matching package source and their candidate versions agree, install/update the matching kernel and device tree:

sudo apt install --reinstall linux-image-vendor-seeed-rk3576 linux-dtb-vendor-seeed-rk3576
sudo reboot

This driver maintenance changes the boot kernel and disconnects SSH. After reconnecting, repeat Section 1.2; a successful package command is not proof that the driver works.

If those packages are unavailable, obtain the vendor image or BSP for this exact board model. Do not install another board's kernel, copy rknpu.ko at random, or use RK182x DKMS packages. For a custom BSP, enable CONFIG_ROCKCHIP_RKNPU=y in the matching board kernel, retain the correct NPU device tree, and follow that BSP's build/deployment steps. Kernel and device-tree setup depends on the board and cannot be supplied by Toolkit's pip install.

#2. Install and verify Runtime and Toolkit Lite2

#2.1 What the three components do

On the PC, import from rknn.api import RKNN; on the board, import Lite2 with from rknnlite.api import RKNNLite. Lite2 runs inference but does not convert ONNX to RKNN; the C++ example does not depend on Lite2.

This guide transfers files with SCP and runs them locally on the board, without PC-to-board USB inference; rknn_server is not needed.

#2.2 Get fixed revisions of the official repositories on the PC

Sections 3 and 4 reuse these directories; clone them once in a new working directory.

On PC:

sudo apt update
sudo apt install git wget cmake make gcc g++ openssh-client
mkdir -p ~/RKNN2_Project
cd ~/RKNN2_Project

git clone https://github.com/airockchip/rknn-toolkit2.git
git -C rknn-toolkit2 checkout 59a913d172e7f5ff03c9076e2ec7b1b1288ffd08

git clone https://github.com/airockchip/rknn_model_zoo.git
git -C rknn_model_zoo checkout bad6c7334531becaf90a561988519b7bec34d0ab

These are the commits used in this test; Model Zoo corresponds to v2.3.2. Pin the code and packages so future repository updates do not change command paths.

#2.3 Install the board Runtime

On PC:

ssh rk3576@BOARD_IP 'mkdir -p ~/RKNN2_Project/packages'

scp ~/RKNN2_Project/rknn-toolkit2/rknpu2/runtime/Linux/librknn_api/aarch64/librknnrt.so \
  rk3576@BOARD_IP:~/RKNN2_Project/packages/

On board: Back up any existing library before installing the new one.

sudo apt update
sudo apt install binutils

if [ -e /usr/lib/librknnrt.so ]; then
  sudo cp -a /usr/lib/librknnrt.so "/usr/lib/librknnrt.so.bak-$(date +%Y%m%d-%H%M%S)"
fi
sudo install -m 0644 ~/RKNN2_Project/packages/librknnrt.so /usr/lib/librknnrt.so
sudo ldconfig
strings /usr/lib/librknnrt.so | grep 'librknnrt version'

The version string in this package is:

librknnrt version: 2.3.2 (429f97ae6b@2025-04-09T09:09:27)

Lite2 requires this step. The Lite2 version tested here looks for /usr/lib/librknnrt.so; setting LD_LIBRARY_PATH only for the Demo directory does not replace its system-library check. On RK3588, the missing file produced Can not find dynamic library on RK3588!; installing it resolved the error.

#2.4 Install Toolkit Lite2 on the board

The board runs Debian 12 with Python 3.11, so use the cp311 aarch64 wheel.

On PC:

scp ~/RKNN2_Project/rknn-toolkit2/rknn-toolkit-lite2/packages/rknn_toolkit_lite2-2.3.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl \
  rk3576@BOARD_IP:~/RKNN2_Project/packages/

On board:

sudo apt install python3.11-venv
python3 -m venv ~/venvs/rknn-lite2
source ~/venvs/rknn-lite2/bin/activate

python -m pip install \
  ~/RKNN2_Project/packages/rknn_toolkit_lite2-2.3.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
python -m pip check
python -c "from rknnlite.api import RKNNLite; print('RKNN Toolkit Lite2 import OK')"

Actual results of the two checks:

No broken requirements found.
RKNN Toolkit Lite2 import OK

Do not copy the PC's x86_64 wheel to the board. If you see ensurepip is not available, install python3.11-venv and recreate the virtual environment. A successful import only verifies the Python package; Section 4 checks actual NPU inference.

#3. Configure Toolkit on Ubuntu PC and convert YOLO11n

#3.1 Create a Python 3.11 environment on the PC

On PC: These commands install Miniforge on a new PC. If Conda is already installed, use it to create the same environment.

mkdir -p ~/Downloads
cd ~/Downloads
wget -c https://github.com/conda-forge/miniforge/releases/download/25.3.0-1/Miniforge3-25.3.0-1-Linux-x86_64.sh
bash Miniforge3-25.3.0-1-Linux-x86_64.sh -b -p "$HOME/miniforge3"
source ~/miniforge3/etc/profile.d/conda.sh
conda create -n rknn2 python=3.11 -y
conda activate rknn2

In each new PC terminal, run source ~/miniforge3/etc/profile.d/conda.sh and conda activate rknn2 before the subsequent Python commands.

#3.2 Install Toolkit2 2.3.2

On PC:

cd ~/RKNN2_Project/rknn-toolkit2/rknn-toolkit2
python -m pip install \
  -r packages/x86_64/requirements_cp311-2.3.2.txt \
  packages/x86_64/rknn_toolkit2-2.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl \
  'onnx==1.16.1' 'protobuf==4.25.4'

python -m pip check
python -m pip show rknn-toolkit2
python -c "from rknn.api import RKNN; print('RKNN Toolkit2 import OK')"

The conversion environment tested here used Python 3.11.14 and Toolkit2 2.3.2; after dependency fixes, pip check returned No broken requirements found..

Pinning ONNX and protobuf explicitly avoids dependencies left by older projects: conversion uses ONNX 1.16.1 and protobuf 4.25.4 in an isolated environment without changing the original Conda environment.

#3.3 Download the officially adapted YOLO11n

On PC:

cd ~/RKNN2_Project/rknn_model_zoo/examples/yolo11/model
bash download_model.sh
ls -lh yolo11n.onnx

The downloaded yolo11n.onnx was 10,527,859 bytes.

Use this Rockchip-adapted ONNX. Its output layout matches the official C++ post-processing; do not replace it with an arbitrary original Ultralytics export. For a custom-trained model, follow the official YOLO11 export instructions.

#3.4 Convert to an RK3576 INT8 model

On PC:

conda activate rknn2
cd ~/RKNN2_Project/rknn_model_zoo
export PYTHONPATH="$PWD${PYTHONPATH:+:$PYTHONPATH}"
ls datasets/COCO/coco_subset_20.txt
ls datasets/COCO/subset

cd examples/yolo11/python
python convert.py ../model/yolo11n.onnx rk3576 i8 ../model/yolo11n_rk3576.rknn
ls -lh ../model/yolo11n_rk3576.rknn

The four arguments are the ONNX path, target platform, quantization type, and output path. This page must use rk3576.

The conversion script uses datasets/COCO/coco_subset_20.txt from the repository for calibration. Run it from examples/yolo11/python so the relative paths resolve. This small dataset is for an initial test; calibrate and assess a production model using data representative of real scenarios.

Excerpt from the actual conversion output:

I rknn-toolkit2 version: 2.3.2
...
--> Building model
...
I rknn building ...
I rknn building done.
done
--> Export rknn model
done

The first log line may include internal build information; success means the target file was generated. A notice that the default input/output types changed to int8 is normal for this quantization script. Let the matching C++ example handle model input/output; do not arbitrarily convert the image to signed values.

This RKNN2 example needs only one .rknn model file on the board.

#4. Build, deploy with SCP, and run

#4.1 Prepare the ARM64 cross-compiler

On PC:

cd ~/RKNN2_Project
wget -c https://dn.odroid.com/compiler/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu.tar
tar -xf gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu.tar

export GCC_COMPILER="$HOME/RKNN2_Project/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu"
"${GCC_COMPILER}-gcc" --version

The tested compiler was Linaro GCC 6.3.1 20170404. GCC_COMPILER is a prefix without the trailing -gcc; the build script appends it.

#4.2 Build the RK3576 YOLO11 C++ example

On PC:

cd ~/RKNN2_Project/rknn_model_zoo
bash build-linux.sh -t rk3576 -a aarch64 -b Release -d yolo11

Use bash build-linux.sh to avoid Permission denied if the repository script lacks an executable bit. Convert the model before building so the script includes the generated .rknn in the deployment directory.

The deployment directory should look like this; other bundled example programs may remain:

install/rk3576_linux_aarch64/rknn_yolo11_demo/
├── rknn_yolo11_demo
├── lib/
│   ├── librknnrt.so
│   └── librga.so
└── model/
    ├── yolo11n_rk3576.rknn
    ├── bus.jpg
    └── coco_80_labels_list.txt

If you convert both platforms in one repository, the install directory may contain two .rknn files. Explicitly select the file with the rk3576 suffix when running.

#4.3 Deploy the entire directory with SCP

On PC:

ssh rk3576@BOARD_IP 'mkdir -p ~/RKNN2_Project'
cd ~/RKNN2_Project/rknn_model_zoo
scp -r install/rk3576_linux_aarch64/rknn_yolo11_demo \
  rk3576@BOARD_IP:~/RKNN2_Project/

Copy both lib/ and model/, not just the executable. The example bundles its Runtime under lib/; the version tested here was 2.3.2.

#4.4 Run object detection on the board

On board:

cd ~/RKNN2_Project/rknn_yolo11_demo
sudo env LD_LIBRARY_PATH="$PWD/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" \
  ./rknn_yolo11_demo model/yolo11n_rk3576.rknn model/bus.jpg

Excerpt from the actual RK3576 output:

model input num: 1, output num: 9
model is NHWC input fmt
model input height=640, width=640, channel=3
...
rknn_run
bus @ (95 136 553 438) 0.944
person @ (108 236 222 535) 0.898
person @ (212 240 284 509) 0.835
person @ (476 229 559 522) 0.831
person @ (79 358 117 515) 0.452
write_image path: out.png width=640 height=640 channel=3 ...

Success means the process exits normally, reports bus and person detections, and creates out.png in the current directory. Coordinates and scores may vary slightly with tool versions and quantization. Stable FPS was not measured; this functional check is not a performance benchmark.

On PC, retrieve the image:

mkdir -p ~/RKNN2_Project/results
scp rk3576@BOARD_IP:~/RKNN2_Project/rknn_yolo11_demo/out.png \
  ~/RKNN2_Project/results/yolo11n-rk3576-out.png

Open the image on the PC to inspect detection boxes. If a display is attached to the board, you can also open it in the board's graphical interface.

#4.5 Verify that Lite2 runs the same model

After C++ detection, use the same model to test the Python inference path.

On board:

cd ~/RKNN2_Project/rknn_yolo11_demo
sudo "$HOME/venvs/rknn-lite2/bin/python" - <<'PYCODE'
import numpy as np
from rknnlite.api import RKNNLite

rknn = RKNNLite()
try:
    ret = rknn.load_rknn('model/yolo11n_rk3576.rknn')
    if ret != 0:
        raise RuntimeError(f'load_rknn failed: {ret}')
    ret = rknn.init_runtime()
    if ret != 0:
        raise RuntimeError(f'init_runtime failed: {ret}')
    outputs = rknn.inference(inputs=[np.zeros((1, 640, 640, 3), dtype=np.uint8)])
    if outputs is None:
        raise RuntimeError('inference failed')
    print('Lite2 inference OK; output count:', len(outputs))
finally:
    rknn.release()
PYCODE

Actual result:

Lite2 inference OK; output count: 9

Use the virtual environment Python by absolute path so sudo python3 does not accidentally call the system Python without Lite2.

#4.6 Troubleshooting and acceptance checks

After completion, the NPU driver should be queryable, Runtime/Lite2 usable, PC conversion successful, the C++ program able to create a detection image, and Lite2 able to return nine output tensors.

#4.7 References

• Official RKNN-Toolkit2 repository: Toolkit, Runtime, Lite2, and documentation.
• RKNN SDK V2.3.2 quick-start guide: This guide uses SSH/SCP for connection and transfer.
• YOLO11 example verified here: ONNX download, conversion script, and C++ post-processing.
• Seeed Armbian extension and package source: Source of matching reComputer kernel and device-tree packages.