CV / ViTPose

ViTPose-Small

reComputer R Series (CM5 + Hailo-10H)

Single-person 2D pose estimation with 17 COCO keypoints using ViTPose-Small on reComputer R Series with Hailo-8 or Hailo-10H.

105 downloads

Size

31 MB

Memory

4GB+

Precision

Hailo HEF / HailoRT 4.23.x (Hailo-8) / 5.1.1 (Hailo-10H)

Choose the device you're using, the set up guide and documentation will update accordingly.

Platform

Inference Engine

Getting Started

Deploy

sudo docker run --rm \
  --name cm5-hailo10h-vit-pose-small \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  -v /usr/lib/libhailort.so.5.1.1:/usr/lib/libhailort.so.5.1.1:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo10h-cv/vit_pose_small:latest \
  python web_detection.py --model_path model/vit_pose_small.hef --video_path video/test.mp4

REST API

Use the REST API to run inference. Copy the commands below.

Curl

curl -X POST "http://<Board_IP>:8000/api/models/vit_pose_small/predict" \
  -F "file=@test.jpg"

Python

import requests
response = requests.post(
    "http://<Board_IP>:8000/api/models/vit_pose_small/predict",
    files={"file": open("test.jpg", "rb")},
    timeout=30,
)
print(response.json())

Model Details

reComputer R Series (CM5 + Hailo-10H)

ViTPose-Small on reComputer R Series (CM5 + Hailo-10H)

ViTPose-Small performs single-person 2D pose estimation: the HEF outputs a 64x48x17 heatmap, the application takes the argmax per channel, scales the coordinates to the 256x192 network input and restores them to the frame before drawing the 17 COCO keypoints.

This page targets reComputer R Series (CM5 + Hailo-10H) with a PCIe Hailo-10H accelerator.

Model Info

Property	Value
Architecture	ViTPose-Small (Vision Transformer backbone)
Task	Single-person pose estimation (17 COCO keypoints)
Input	256x192x3 RGB (letterbox, normalization compiled into the HEF)
Output	Heatmap 64x48x17 (argmax decode, no on-chip NMS)
Parameters	24.29M
Operations	17.17G
HEF	Hailo Model Zoo v5.4.0, Hailo-10H (24,653,824 bytes)

Hardware and Host Setup

Item	Value
Board	reComputer R Series with Raspberry Pi CM5
Accelerator	Hailo-10H over PCIe, exposed as `/dev/hailo0`
Runtime	HailoRT 5.1.1
Python in container	3.13, aarch64

Install the Hailo-10H packages on the host and confirm that the driver and device are ready before starting the container:

bash

sudo apt update
sudo apt install hailo-h10-all -y
sudo reboot

# After reboot
hailortcli --version
hailortcli fw-control identify
ls -l /dev/hailo0

Run With Demo Video

bash

sudo docker run --rm \
  --name cm5-hailo10h-vit-pose-small \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  -v /usr/lib/libhailort.so.5.1.1:/usr/lib/libhailort.so.5.1.1:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo10h-cv/vit_pose_small:latest \
  python web_detection.py --model_path model/vit_pose_small.hef --video_path video/test.mp4

Open http://<Board_IP>:8000 to view the preview: the COCO skeleton is drawn over the person in the frame.

USB Camera Mode

bash

sudo docker run --rm \
  --name cm5-hailo10h-vit-pose-small \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  --device /dev/video0:/dev/video0 \
  -v /usr/lib/libhailort.so.5.1.1:/usr/lib/libhailort.so.5.1.1:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo10h-cv/vit_pose_small:latest \
  python web_detection.py --model_path model/vit_pose_small.hef --camera_id 0

REST API

text

POST http://<Board_IP>:8000/api/models/vit_pose_small/predict

bash

curl -X POST "http://<Board_IP>:8000/api/models/vit_pose_small/predict" \
  -F "file=@test.jpg"

Endpoint	Method	Purpose
`/`	GET	Web preview UI
`/api/models/vit_pose_small/predict`	POST	17 COCO keypoints (JSON)
`/api/video_feed`	GET	MJPEG preview stream
`/api/config`	GET / POST	Read or change runtime thresholds
`/api/video/upload`	POST	Upload a source video
`/api/video/analyze`	POST	Start offline video analysis
`/api/video/status`	GET	Read analysis progress
`/api/video/download/{filename}`	GET	Download the annotated result

Keypoints below the confidence threshold (0.30) are dropped from the skeleton overlay.

Implementation Notes

Single-person model: the network expects the person to be centred in the crop, so multi-person scenes need a detector to crop each person first.
Pre-processing letterboxes to 256x192 with black padding, converts BGR to RGB and feeds raw uint8; the ImageNet normalization (mean [123.675, 116.28, 103.53], std [58.395, 57.12, 57.375] on RGB) is compiled into the HEF.
Post-processing takes the argmax per heatmap channel to get (x, y) on the 64x48 map, scales it to the 256x192 input and un-letterboxes it to the frame. There is no on-chip NMS and no DARK sub-pixel refinement.
The first inference logs the raw output shape once ([ViTPose] raw output shape=...) so the layout can be confirmed on hardware.

Development Notes

Source module: src/hailo10h_vit_pose_small/
Dockerfile: docker/hailo10h/vit_pose_small.dockerfile
Container: ghcr.io/seeed-projects/recomputer-hailo10h-cv/vit_pose_small:latest
The module uses the HailoRT 5.1.1 InferModel API and a Hailo-10H-specific HEF.
Unit tests run without a device: python -m unittest discover -s tests -v.

Inputs and Outputs

Input: image, video, or USB camera frame. Output: 17 COCO keypoints (x, y, score) with skeleton overlay on MJPEG preview.