inteintegrity2026-11-14

YOLO11 Warehouse Safety on reComputer RK3576

Detect forklifts and pedestrians with YOLO11 and ByteTrack on the reComputer RK3576 NPU. Includes local Web UI, red danger alerts, and RKNN models.

CVWarehouse SafetyreComputer RKGitHub
Forklift and pedestrian safety monitoring demo on reComputer RK3576

In warehouse aisles shared by forklifts and pedestrians, spotting a close approach is an important part of safety monitoring. This project brings vehicle detection, person tracking, and zone alerts into a local Web UI, showing the operator who is approaching a forklift and the current risk state.

This project runs a warehouse-trained YOLO11n model on the reComputer RK3576 NPU, uses ByteTrack to follow forklifts and people, and shows CAUTION and red DANGER alerts when a pedestrian approaches the projected vehicle zones. A browser dashboard provides an annotated feed, current risk pairs, and event downloads. Inference and analysis run locally on the board.

Source code: rk3576-warehouse-safety-monitor

Forklift and Pedestrian Safety Features

FeatureBehavior
Forklift and person detectionA custom two-class YOLO11n model runs on the RK3576 NPU.
Persistent identitiesByteTrack assigns IDs across frames.
Operator exclusionA bounding-box heuristic excludes the forklift operator from pedestrian risk pairing.
Graduated alertsGreen monitoring, yellow CAUTION, and red DANGER indicate projected-zone proximity.
Local Web UIView annotated frames, current pairs, and completed events in a browser.
Exportable resultsSave annotated MP4, event CSV, run-summary JSON, and per-frame traces.
Showcase renderingAdd translucent glowing floor zones to recorded inference results on a desktop.

The sample is a 10-second, 1280 × 720 video at 24 FPS. The close-approach moments around 5 and 8 seconds trigger red danger alerts. The driver is labeled separately as OPERATOR.

How It Works

text
Video file / USB camera / RTSP
  → resize and letterbox to 640 × 640 RGB
  → YOLO11n RKNN inference on the RK3576 NPU
  → CPU detection decoding and NMS
  → CPU ByteTrack tracking
  → operator exclusion and pairwise zone analysis
  → Web UI, annotated video, and event logs

YOLO11n detects forklifts and people, while ByteTrack maintains their identities across frames. The system pairs each forklift with nearby pedestrians outside the vehicle, updates alerts from their positions relative to the zones, and records continuous alerts as exportable events. Detection runs on the NPU; tracking, alert logic, drawing, and recording run on the CPU.

How Proximity Alerts Work

The system builds projected image-space zones around the forklift's detection box and continuously checks each pedestrian's position relative to those zones. The point on the pedestrian's bottom box edge closest to the forklift center determines whether a proximity warning or danger alert is triggered.

Approaching the warning region displays yellow CAUTION. Entering the inner danger region raises red DANGER. The zones visualize proximity in the image; they do not report uncalibrated distances in meters.

What the Colors Mean

StateDisplayTrigger
MonitoringGreen person boxOutside the warning region.
CAUTIONYellow person box and caution messageNear the projected vehicle zone.
DANGERRed person box, danger message, and red border in the showcaseEnters the inner projected zone.

The ellipse is centered at the forklift box's bottom center. Its base horizontal radius is 2.4 box widths and vertical radius is 0.4 box heights. In normalized ellipse coordinates, danger starts at radius 0.85 and caution at 1.05. Danger clears beyond 0.93, and caution clears beyond 1.20, reducing boundary flicker. The outer green display ring has radius 1.35.

These are configurable image-region settings. They do not describe measured floor distances or the true dimensions of the forks and load.

The prepared project includes models/warehouse_yolo11_rk3576_int8.rknn and models/warehouse_yolo11_rk3576_fp16.rknn, with checksums in models/manifest.json. Their class order is 0: forklift, 1: person. A stock COCO detector cannot replace them because COCO has no forklift class.

Run on reComputer RK3576

Use ARM64 Linux with Python 3.11 and a working RK3576 NPU driver/runtime. Clone the project, prepare its environment, and start the dashboard:

bash
git clone https://github.com/inteintegrity/rk3576-warehouse-safety-monitor.git
cd rk3576-warehouse-safety-monitor
bash run.sh setup
bash run.sh web

Open http://<board-ip>:8088 from a computer on the same trusted LAN, replacing <board-ip> with the board's address. In the dashboard:

  1. Keep data/test_clip_nearmiss.mp4 as the source for the supplied sample.
  2. Select Start analysis and watch the feed and current risk pairs.
  3. Select Stop to end analysis early, or let the sample finish.
  4. Download the annotated MP4, event CSV, or summary JSON after completion.

Press Ctrl+C in the terminal to stop the server. To use another port, run bash run.sh web --port 8089. The input field accepts an existing video path, USB index, or RTSP URL; it does not have an upload control. File-source inference and preview were verified; USB/RTSP still require testing with your cameras. The single-operator dashboard has no authentication and is intended for a trusted LAN.

For analysis without the dashboard:

bash
bash run.sh video

Results go to outputs/demo: annotated.mp4, events.csv, summary.json, and frame-trace.jsonl. Add --source /path/to/video.mp4 to use another video. Events record paired IDs, start/end times, duration, and peak tier. Image-zone mode leaves metric distance empty rather than presenting uncalibrated meters. bash run.sh check checks dependencies and model hashes; bash run.sh --help lists all launcher commands.

RK3576 NPU Performance

The final INT8 CLI run processed all 240 frames without a dashboard stream:

MeasurementResult
End-to-end time, excluding initialization22.805 s
Complete pipeline throughput10.52 FPS
Median NPU API inference, after 15 warmup frames25.551 ms
95th-percentile NPU API inference, after warmup31.269 ms

Pipeline throughput includes decoding, preprocessing, NPU inference, CPU postprocessing, tracking, drawing, trace writing, and MP4 recording. The later Web UI check completed 240 frames at approximately 9.5 FPS, with MJPEG preview and CSV/JSON downloads verified. These are individual demo runs, not sustained performance guarantees.

Make the Glowing Showcase Video

The board's live view uses lightweight zone outlines and colored alert boxes. The translucent red/yellow/green glow is rendered offline from the original video and the board's per-frame results:

bash
bash run.sh render outputs/demo

Run this step in a Bash environment with Pillow, OpenCV, and FFmpeg after copying the sample video and board outputs into the corresponding project directories. Activate the rendering machine's Python environment first. The script handles frame rendering and H.264 MP4 encoding. The exported showcase is 1920 × 1080 at 24 FPS and preserves the source's 10-second timeline. Its playback rate is not a 24 FPS live-inference claim.

Validation and Limits

Eight regression checks passed, covering driver exclusion, DFL decoding, preprocessing, empty detections/logs, timestamp handling, invalid inputs, red alerts at the close-approach frames, and alert hysteresis. Run them with:

bash
bash run.sh test

The original model was trained with 24 labeled frames from the same sample video. The demo also uses sample frames for INT8 calibration, so the clip verifies the pipeline rather than independent detection accuracy. New camera angles, occlusions, load shapes, and operator poses require further validation. The zones are a visual proximity heuristic, not a safety-certified collision prediction system.