MSPN RegNetX-800MF on reComputer R Series (CM5 + Hailo-8)

MSPN (Multi-Stage Pose Network) with a RegNetX-800MF backbone performs single-person pose estimation with 17 COCO keypoints. The HEF outputs a 64x48x17 heatmap; the app does argmax per channel, scales to input space, and draws the COCO skeleton.

This page targets reComputer R Series (CM5 + Hailo-8) with a PCIe Hailo-8 accelerator.

Model Info

Hardware and Host Setup

The Hailo-8 driver, firmware, host libhailort.so, and the Python wheel in the image must use matching HailoRT major/minor versions (4.23.x).

sudo apt update
sudo apt install hailort hailort-pcie-driver python3-hailort
sudo reboot

hailortcli --version
hailortcli fw-control identify
ls -l /dev/hailo0

Run With Demo Video

sudo docker run --rm \
  --name cm5-hailo8-mspn-regnetx-800mf \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  -v /usr/lib/libhailort.so.4.23.0:/usr/lib/libhailort.so.4.23.0:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo8-cv/mspn_regnetx_800mf:latest \
  python web_detection.py --model_path model/mspn_regnetx_800mf.hef --video_path video/test.mp4

Open http://<Board_IP>:8000 to view the web preview with the keypoint skeleton overlay.

USB Camera Mode

sudo docker run --rm \
  --name cm5-hailo8-mspn-regnetx-800mf \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  --device /dev/video0:/dev/video0 \
  -v /usr/lib/libhailort.so.4.23.0:/usr/lib/libhailort.so.4.23.0:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo8-cv/mspn_regnetx_800mf:latest \
  python web_detection.py --model_path model/mspn_regnetx_800mf.hef --camera_id 0

REST API

curl -X POST "http://<Board_IP>:8000/api/models/mspn_regnetx_800mf/predict" \
  -F "file=@test.jpg"

Implementation Details

• Source module: src/rpi5_hailo8_mspn_regnetx_800mf/
• Dockerfile: docker/hailo8/mspn_regnetx_800mf.dockerfile
• normalize_in_net with ImageNet RGB mean/std; no input_conversion — feed raw uint8 RGB after letterboxing.
• Decode: argmax per heatmap channel → (x, y) in 64x48 space → scale to 256x192 → un-letterbox to the original frame. The raw vstream layout is logged once at first inference.
• Single-person model: it assumes one centered person. For multiple people, run a detector first and crop each person.
• The Model Zoo reference adds a 5x5 Gaussian blur and a quarter-pixel shift; this demo skips both.

Troubleshooting

• HEF fails to load: confirm HailoRT is 4.23.x and the mounted libhailort.so matches the host driver.
• No /dev/hailo0: check lsmod | grep hailo and reinstall hailort-pcie-driver.
• No skeleton drawn: lower the keypoint confidence slider (default 0.30).
• Wrong keypoint positions: confirm the printed input/output shapes match the 256x192 input and 64x48x17 heatmap.