CV / STARK

STARK-Lightning

reComputer R Series (CM5 + Hailo-8)

Single-object tracking with STARK-Lightning on reComputer R Series with Hailo-8.

3 downloads
Size
4.2 MB
Memory
4GB+
Precision
Hailo HEF

Choose the device you're using, the set up guide and documentation will update accordingly.

Getting Started

Deploy
sudo docker run --rm \
  --name cm5-hailo-stark-lightning \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  -v /usr/lib/libhailort.so.4.23.0:/usr/lib/libhailort.so.4.23.0:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo8-cv/stark_lightning_repvgg_a0:latest

REST API

Use the REST API to run inference. Copy the commands below.

Curl
curl http://localhost:8080/v1/chat/completions -d '{
  "model": "stark-lightning",
  "messages": [{"role": "user", "content": "Hello"}]
}'
Python
import requests

resp = requests.post(
    "http://localhost:8080/v1/chat/completions",
    json={"model": "stark-lightning", "messages": [{"role": "user", "content": "Hello"}]},
)
print(resp.json())

Official Benchmarks

These figures come from Rockchip’s official benchmarks. Performance in your application may vary.

stark_lightning_repvgg_a0

reComputer R Series (CM5 + Hailo-8) · 320x320 · Hailo HEF

Float Success-plot AUC
56.9
Hardware Success-plot AUC
55.2
FPS (batch 1)
1074
Parameters (M)
4.62
Operations (G)
3.48

Model Details

STARK-Lightning Single-Object Tracking on reComputer R Series (CM5 + Hailo-8)

STARK-Lightning tracks one target through a video on reComputer R Series with CM5 and Hailo-8. The official Hailo-8 HEF takes a 128x128 template crop and a 320x320 search crop. The application initializes the template from a box selected in the first frame and updates the box on every later frame.

Hardware and Host Setup

ItemValue
BoardreComputer R Series with Raspberry Pi CM5
AcceleratorHailo-8 M.2 over PCIe, /dev/hailo0
OSRaspberry Pi OS Bookworm, aarch64
Host driverhailo-all apt package
HailoRT4.23.x; driver, firmware, native library, and wheel must share major.minor
bash
sudo apt update
sudo apt install hailo-all
sudo reboot

# After reboot
hailortcli fw-control identify
ls /dev/hailo0

# Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh --mirror Aliyun
sudo systemctl enable docker
sudo systemctl start docker

Run With Docker

Build the image from the module directory:

bash
cd src/rpi5_hailo8_stark_lightning_repvgg_a0
sudo docker build -f ../../docker/hailo8/stark_lightning_repvgg_a0.dockerfile \
  -t rpi5-hailo8-stark:latest .

Run the WebUI:

bash
sudo docker run --rm --privileged --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  -v /usr/lib/libhailort.so.4.23.0:/usr/lib/libhailort.so.4.23.0:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo8-cv/stark_lightning_repvgg_a0:latest

Open http://<BOARD_IP>:8000, upload a video, drag a box around the target in the first frame, and click Run tracking.

Model and Tracking Pipeline

ItemValue
HEFstark_lightning_repvgg_a0.hef, 4,425,622 B, Model Zoo v2.20.0
HEF SHA2567319ef5e0016e85510a71a86e8231a1fb0558a6475fa041e08919cc960c03299
Template input128x128x3 RGB
Search input320x320x3 RGB
Output4 normalized xyxy coordinates in the search crop
Parameters / operations4.62M / 3.48G

The first frame crops a square region of 2x the target area around the selected box and resizes it to 128x128. Each later frame crops a 5x search region around the previous box center, resizes it to 320x320, and runs both inputs through the HEF. The two 20x20 corner heatmaps are decoded with a per-channel softmax and soft-argmax on the 320x320 grid; the resulting 4 values are scaled by the search-crop ratio, offset by the crop position, and clipped to the frame.

The HEF applies normalization in-network (mean [123.675, 116.28, 103.53], std [58.395, 57.12, 57.375], RGB order), so the application feeds raw RGB pixels as float32.

HTTP API

  • POST /api/video/upload — upload a video.
  • GET /api/video/first_frame?filename=... — JPEG of the first frame.
  • POST /api/video/analyze — start tracking with filename, x, y, w, h.
  • POST /api/video/analyze/stop — request cancellation.
  • GET /api/video/status — progress and output name.
  • GET /api/video/download/{filename} — download the annotated MP4.
bash
curl -X POST "http://<BOARD_IP>:8000/api/video/analyze" \
  -F "filename=demo.mp4" -F "x=420" -F "y=180" -F "w=160" -F "h=240"

Performance and Validation Status

Official Hailo-8 results on LaSOT: 56.9 Success-plot AUC at full precision and 55.2 on hardware, with 1074 FPS at batch size 1. These are Model Zoo figures, not measurements of this WebUI pipeline.

Static checks, synthetic crop/mapping tests, and a shape test pass. Board loading, input/output layout, and real tracking accuracy are still pending; see the module TEST_REPORT.md. Do not present this page as hardware-validated until those checks are complete.

Limitations

  • The target box must be selected in the first frame; there is no re-initialization or multi-object tracking.
  • Tracking runs offline over uploaded video; there is no live camera preview.
  • The module has not been run on physical Hailo-8 hardware here.
  • The Web service targets a trusted LAN and has no authentication or TLS.
  • Output videos have no audio; odd dimensions may be cropped for encoding.

Inputs and Outputs

Input: a video and one target box on the first frame. Output: a tracked box (x, y, w, h) per frame and an annotated MP4.