CV / CAS-ViT

CAS-ViT

reComputer R Series (CM5 + Hailo-8)

CAS-ViT-S/M/T ImageNet classification on reComputer R Series with Hailo-8 or Hailo-10H.

13 downloads
Size
12.7 MB
Memory
4GB+
Precision
Hailo HEF

Choose the device you're using, the set up guide and documentation will update accordingly.

Getting Started

Deploy
sudo docker run --rm \
  --name pi5-hailo8-cas-vit-s \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  -v /usr/lib/libhailort.so.4.23.0:/usr/lib/libhailort.so.4.23.0:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo8-cv/cas_vit_s:latest \
  python web_detection.py --model_path model/cas_vit_s.hef --video_path video/test.mp4

Model Details

reComputer R Series (CM5 + Hailo-8)

CAS-ViT-S/M/T ImageNet Classification on reComputer R Series (CM5 + Hailo-8)

CAS-ViT-S/M/T classify 1000 ImageNet classes on Hailo-8 through HailoRT 4.23. The HEF takes a 384×384 BGR frame, performs bgr_to_rgb and ImageNet normalization on-chip, and returns 1000 scores; the host applies softmax, smooths the scores over time and overlays a Top-K panel on the MJPEG preview.

Model Info

VariantInputPackaged HEFTop-1 float / Hailo-8Params / OpsContainer
CAS-ViT-S384×384×3 BGR12.7 MB79.9 / 79.75.5M / 5.4Gcas_vit_s
CAS-ViT-M384×384×3 BGR18.5 MB81.2 / 80.912.42M / 10.89Gcas_vit_m
CAS-ViT-T384×384×3 BGR26.4 MB81.9 / 81.621.76M / 20.85Gcas_vit_t

Hardware and Host Setup

ItemValue
BoardreComputer R Series with Raspberry Pi CM5
AcceleratorHailo-8 over PCIe, exposed as /dev/hailo0
RuntimeHailoRT 4.23.x (host / container must share major.minor)
Python in container3.11, aarch64
bash
sudo apt update
sudo apt install hailo-all -y
sudo reboot

# After reboot
hailortcli fw-control identify
ls /dev/hailo0

# Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh --mirror Aliyun
sudo systemctl enable docker
sudo systemctl start docker

Run With Demo Video

Select the command matching the model variant.

CAS-ViT-S

bash
sudo docker run --rm \
  --name pi5-hailo8-cas-vit-s \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  -v /usr/lib/libhailort.so.4.23.0:/usr/lib/libhailort.so.4.23.0:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo8-cv/cas_vit_s:latest \
  python web_detection.py --model_path model/cas_vit_s.hef --video_path video/test.mp4

CAS-ViT-M

bash
sudo docker run --rm \
  --name pi5-hailo8-cas-vit-m \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  -v /usr/lib/libhailort.so.4.23.0:/usr/lib/libhailort.so.4.23.0:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo8-cv/cas_vit_m:latest \
  python web_detection.py --model_path model/cas_vit_m.hef --video_path video/test.mp4

CAS-ViT-T

bash
sudo docker run --rm \
  --name pi5-hailo8-cas-vit-t \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  -v /usr/lib/libhailort.so.4.23.0:/usr/lib/libhailort.so.4.23.0:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo8-cv/cas_vit_t:latest \
  python web_detection.py --model_path model/cas_vit_t.hef --video_path video/test.mp4

Open http://<Board_IP>:8000 to view the Top-K overlay. The bundled video/test.mp4 is Hailo's official classifier demo clip (example.mp4, a street scene with cars and people).

USB Camera Mode

bash
sudo docker run --rm \
  --name pi5-hailo8-cas-vit-m \
  --privileged \
  --net=host \
  -e PYTHONUNBUFFERED=1 \
  --device /dev/hailo0:/dev/hailo0 \
  --device /dev/video0:/dev/video0 \
  -v /usr/lib/libhailort.so.4.23.0:/usr/lib/libhailort.so.4.23.0:ro \
  -v /usr/lib/libhailort.so:/usr/lib/libhailort.so:ro \
  ghcr.io/seeed-projects/recomputer-hailo8-cv/cas_vit_s:latest \
  python web_detection.py --model_path model/cas_vit_s.hef --camera_id 0

Swap cas_vit_s for cas_vit_m or cas_vit_t to run another variant.

REST API

EndpointMethod
POST /api/models/cas_vit_s/predictTop-K classification (JSON)
POST /api/models/cas_vit_m/predictTop-K classification (JSON)
POST /api/models/cas_vit_t/predictTop-K classification (JSON)
bash
curl -X POST "http://<Board_IP>:8000/api/models/cas_vit_m/predict" \
  -F "file=@test.jpg" -F "conf=0.30" -F "iou=5"

Returns predictions entries with the ImageNet class name, a short label and the confidence. iou is reused as the Top-K count for API parity with the detection modules.

Implementation Notes

  • The HEF exposes a single 1000-way score vector; the ALLS recipe contains no softmax or argmax, so the host runs softmax and Top-K.
  • Pre-processing is a plain resize to 384×384 and keeps BGR. The CAS-ViT ALLS recipe performs bgr_to_rgb and the ImageNet mean/std normalization inside the HEF. No letterbox is used, and the app must not convert to RGB.
  • The live preview smooths the probability vector (0.6 previous + 0.4 current) to stop the label flickering, then draws the Top-K panel.
  • The HEF input size is read at startup (384×384).

Development Notes

  • Source: src/rpi5_hailo8_cas_vit_{s,m,t}/
  • Dockerfile: docker/hailo8/cas_vit_{s,m,t}.dockerfile
  • Container: ghcr.io/seeed-projects/recomputer-hailo8-cv/cas_vit_{s,m,t}:latest
  • HEF: Hailo Model Zoo v2.20.0 (hailo8), size and SHA256 recorded in the module's model/README.md
  • Demo video: https://hailo-csdata.s3.eu-west-2.amazonaws.com/resources/video/example.mp4 (from hailo-ai/hailo-apps, MIT)

Inputs and Outputs

Input: image, video frame, or camera frame. Output: Top-K ImageNet labels and confidence scores.