YOLOv8 Real-time Inference CPP Project with reComputer RK3576

A multi-source input, multi-stream output YOLOv8 real-time inference project based on RK3576 / RKNN / MPP / RGA / ZLMediaKit.

Current project supports:

• V4L2 camera input
• RTSP network video input
• localvideo local video file looping input
• Three RTSP output streams: main stream / sub stream / AI debug stream
• YOLOv8 RKNN inference, bounding box overlay, class labels, time and FPS OSD
• libavformat prioritized for RTSP/local compressed stream input, with ffmpeg pipe as a fallback

Feature Overview

• Input Sources

• v4l2: Suitable for USB cameras, local capture

• rtsp: Suitable for IPC/NVR/network video streams

• localvideo: Suitable for local video playback, offline validation, loop inference

• Output Streams

• Main stream: rtsp://<board-ip>:8554/live/camera

• Sub stream: rtsp://<board-ip>:8554/live/camera_sub

• AI debug stream: rtsp://<board-ip>:8554/live/camera_ai

• Use --streams main,sub,ai to output any single, dual, or all three streams simultaneously

• Resolution Strategies

• camera main stream: Proportional scaling + padding (no stretching)

• camera_sub sub stream: Proportional scaling + center cropping (no stretching)

• camera_ai AI debug stream: Proportional scaling + padding (no stretching)

• AI Inference

• Fixed input of 640x640

• Prioritizes RGA letterbox for preprocessing

• Supports detection cache, PTS matching, AI overlay

Deployment Guide

Dependencies

• RK3576 board runtime environment
• RKNN Runtime
• Rockchip MPP
• RGA / librga
• ZLMediaKit runtime library
• FFmpeg runtime library or command-line tools

Build Steps

cd /home/parker/Projects/rk3576_yolov8tortsp_demo
chmod +x scripts/build-linux.sh
./scripts/build-linux.sh

Build outputs:

• Executable: install/bin/rk3576_yolov8tortsp_demo
• Runtime libraries: install/lib
• Documentation: install/docs
• Models directory: install/model

Pre-run Setup

cd install
export LD_LIBRARY_PATH="$(pwd)/lib:${LD_LIBRARY_PATH}"

Model Placement

Place the RKNN models into:

model/
  yolov8n_rk3576.rknn
  yolov8s_rk3576.rknn
  yolov8m_rk3576.rknn

If --model is not explicitly provided, the program will automatically select existing models in the following order:

1. model/yolov8n_rk3576.rknn
2. model/yolov8s_rk3576.rknn
3. model/yolov8m_rk3576.rknn

Usage

V4L2 Camera Input

sudo ./bin/rk3576_yolov8tortsp_demo \
  --source v4l2 \
  --device /dev/video0 \
  --model model/yolov8n_rk3576.rknn \
  --camera-width 1920 \
  --camera-height 1080 \
  --output-width 1280 \
  --output-height 720 \
  --fps 30 \
  --bitrate 4000000 \
  --perf-interval-ms 1000 \
  --rtsp-port 8554 \
  --rtsp-app live \
  --rtsp-stream camera

RTSP Input

sudo ./bin/rk3576_yolov8tortsp_demo \
  --source rtsp \
  --rtsp-input-url "rtsp://192.168.100.101:554/live" \
  --rtsp-username "admin" \
  --rtsp-password "c32bdc3e" \
  --model model/yolov8n_rk3576.rknn \
  --camera-width 1920 \
  --camera-height 1080 \
  --output-width 1280 \
  --output-height 720 \
  --fps 30 \
  --bitrate 4000000 \
  --perf-interval-ms 1000 \
  --rtsp-timeout-ms 5000 \
  --rtsp-reconnect-ms 1000 \
  --rtsp-port 8554 \
  --rtsp-app live \
  --rtsp-stream camera

Local Video Looping Input

sudo ./bin/rk3576_yolov8tortsp_demo \
  --source localvideo \
  --localvideo ./videos/demo.mp4 \
  --model model/yolov8n_rk3576.rknn \
  --camera-width 1920 \
  --camera-height 1080 \
  --output-width 1280 \
  --output-height 720 \
  --fps 30 \
  --bitrate 4000000 \
  --perf-interval-ms 1000 \
  --rtsp-port 8554 \
  --rtsp-app live \
  --rtsp-stream camera

Detailed Logs

Only a startup summary is printed by default; to print detailed runtime logs, add:

--detail-info

Export NPU Input Frames

sudo ./bin/rk3576_yolov8tortsp_demo \
  --source localvideo \
  --localvideo ./videos/demo.mp4 \
  --model model/yolov8n_rk3576.rknn \
  --streams ai \
  --dump-ai-input-dir ./debug_ai_input \
  --dump-ai-input-every 30 \
  --detail-info

Enable Specific Output Streams

--streams ai
--streams main,ai
--streams main,sub
--streams main,sub,ai

Explanation:

• --streams ai: Perform AI inference only and output camera_ai
• --streams main,ai: Output main stream and AI debug stream
• --streams main,sub: Standard dual-stream only, does not load RKNN model
• When omitted, defaults to --streams main,sub,ai

View Help

./bin/rk3576_yolov8tortsp_demo --help

API Reference

Core Command Line Arguments

• --source <v4l2|rtsp|localvideo>: Input source type
• --device <path>: V4L2 device node
• --rtsp-input-url <url>: RTSP address
• --rtsp-username <value>: RTSP username
• --rtsp-password <value>: RTSP password
• --localvideo <path>: Local video file path
• --model <path>: RKNN model path
• --camera-width <value> / --camera-height <value>: Input resolution
• --output-width <value> / --output-height <value>: Main stream output resolution
• --sub-width <value> / --sub-height <value>: Sub stream output resolution
• --fps <value>: Target encoding framerate
• --bitrate <value>: Main stream target bitrate
• --rtsp-port <value>: Local RTSP server port
• --rtsp-app <value>: RTSP app name
• --rtsp-stream <value>: Main stream name
• --sub-stream <value>: Sub stream name
• --ai-stream <value>: AI debug stream name
• --streams <items>: Enable which output streams, supports comma-separated combinations of main, sub, ai
• --rtsp-timeout-ms <value>: RTSP read timeout
• --rtsp-reconnect-ms <value>: RTSP reconnect interval
• --perf-interval-ms <value>: PERF output cycle
• --detail-info: Enable detailed logs
• --help: Print help

PERF Meanings

• process: Image processing time within the branch
• encode: MPP encoding time
• push: RTSP pushing time
• e2e: Total time from start of processing to push completion within the branch
• fps: Actual branch throughput calculated by wall-clock statistical window

Project Structure Overview

rk3576_yolov8tortsp_demo
├── CMakeLists.txt
├── README.md
├── docs
├── include
│   ├── rk3576_demo
│   └── rk3576_yolo_demo
├── model
├── scripts
├── src
│   ├── ai          # RKNN inference and pre/post-processing
│   ├── app         # Application entry and orchestration
│   ├── branch      # 3-branch output, OSD, PERF
│   ├── camera      # V4L2 capture
│   ├── codec       # MPP encode/decode
│   ├── common      # Common types, logger, thread queues
│   ├── pipeline    # V2 distribution and processing pipeline
│   ├── rga         # Image scaling and conversion
│   ├── rtsp        # Local RTSP publishing
│   ├── source      # Input source abstraction and implementation
│   ├── utils
│   └── watermark   # Time/FPS/Label rendering
└── third_party

Development Guide

1. Extending Input Sources

• Unified interface at include/rk3576_yolo_demo/source/i_source.hpp
• Factory entry at src/source/source_factory.cpp
• New input sources should implement:
  • Open()
  • Close()
  • ReadPacket()
  • Describe()

2. Extending Branch Outputs

• Branch implementation located at src/branch/branch_output.cpp
• Currently supports three geometry strategies:
  • stretch
  • letterbox
  • center_crop
• Geometric relations are calculated uniformly in ComputeFrameTransform(), facilitating reuse for output scaling and AI box mapping

3. Extending AI Inference

• RKNN entry at src/ai/yolov8_engine.cpp
• Preprocessing prioritizes RGA
• Detection results enter the DetectionFrame cache, then are overlaid by the AI branch based on PTS matching

4. Logging and Debugging

• Prints only startup summary by default
• Detailed runtime logs are enabled via --detail-info
• Logger is at include/rk3576_yolo_demo/common/logger.hpp

Deployment Recommendations

• For production deployment, ensure proper permissions are set for /dev/rga, video devices, and relevant runtime libraries
• It is recommended to package models, runtime libraries, and configurations together with install/
• RTSP input prioritizes libavformat, automatically falling back to ffmpeg pipe in older environments
• For long-term stable operation, it is recommended to use systemd or supervisor for daemon and restart management

Known Limitations

• Box drawing and watermark overlay for camera_ai still consume some CPU overhead
• e2e is not the complete full-link latency; it only represents the time spent within the branch
• localvideo is currently based on the compressed stream reading link, mainly used for offline validation and loop inference

Future Development Suggestions

1. Continue to reduce OSD CPU overhead for camera_ai, potentially introducing tile caching or more efficient blending paths
2. Add a true full-link pipeline_e2e metric, distinct from the current branch e2e
3. Improve playback pacing control in localvideo scenarios to make it closer to real-time sources
4. Add more independent encoding parameter configuration capabilities for main/sub streams
5. If more complex input types are needed, continue to extend the source abstraction layer