CV / YOLO-World

YOLO-World

Change detection categories at runtime with English natural-language prompts without rebuilding the model.

4 downloads

Precision

INT8 detector / FP16 text encoder

Choose the device you're using, the set up guide and documentation will update accordingly.

Platform

Inference Engine

Getting Started

Deploy

sudo docker run --rm --privileged -p 8000:8000 \
  -v /dev/dri/renderD129:/dev/dri/renderD129 \
  -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
  ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yolo_world:latest \
  python3 web_detection.py --platform rk3576 \
  --model_path /app/model/yolo_world_v2s_i8.rknn \
  --text_model /app/model/clip_text_fp16.rknn \
  --text_features /app/model/coco_text_outp.npy \
  --vocab_path /app/model/clip_vocab.txt --class_path /app/model/detect_classes.txt \
  --camera_id -1 --host 0.0.0.0 --port 8000

REST API

Use the REST API to run inference. Copy the commands below.

Curl

curl http://localhost:8080/v1/chat/completions -d '{
  "model": "yolo-world-rknn",
  "messages": [{"role": "user", "content": "Hello"}]
}'

Python

import requests

resp = requests.post(
    "http://localhost:8080/v1/chat/completions",
    json={"model": "yolo-world-rknn", "messages": [{"role": "user", "content": "Hello"}]},
)
print(resp.json())

Model Details

YOLO-World on reComputer RK3576 and RK3588

This project provides open-vocabulary detection from reComputer-RK-CV. Both the INT8 YOLO-World detector and FP16 CLIP text encoder run on the NPU, allowing detection categories to change without rebuilding the detector.

The bundled CLIP model is primarily trained for English. Short English noun phrases such as person, red bus, and traffic light are recommended.

Model information

Component	File
Detector	`/app/model/yolo_world_v2s_i8.rknn`
Text encoder	`/app/model/clip_text_fp16.rknn`
Initial COCO features	`/app/model/coco_text_outp.npy`
BPE vocabulary	`/app/model/clip_vocab.txt`
Default labels	`/app/model/detect_classes.txt`

Run the service

Use the image matching the board and set --platform to the same value.

RK3576

bash

sudo docker run --rm --privileged -p 8000:8000 \
  -v /dev/dri/renderD129:/dev/dri/renderD129 \
  -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
  ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yolo_world:latest \
  python3 web_detection.py --platform rk3576 \
  --model_path /app/model/yolo_world_v2s_i8.rknn \
  --text_model /app/model/clip_text_fp16.rknn \
  --text_features /app/model/coco_text_outp.npy \
  --vocab_path /app/model/clip_vocab.txt \
  --class_path /app/model/detect_classes.txt \
  --video_path /app/video/test.mp4 --host 0.0.0.0 --port 8000

RK3588

bash

sudo docker run --rm --privileged -p 8000:8000 \
  -v /dev/dri/renderD129:/dev/dri/renderD129 \
  -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
  ghcr.io/seeed-projects/recomputer-rk-cv/rk3588-yolo_world:latest \
  python3 web_detection.py --platform rk3588 \
  --model_path /app/model/yolo_world_v2s_i8.rknn \
  --text_model /app/model/clip_text_fp16.rknn \
  --text_features /app/model/coco_text_outp.npy \
  --vocab_path /app/model/clip_vocab.txt \
  --class_path /app/model/detect_classes.txt \
  --video_path /app/video/test.mp4 --host 0.0.0.0 --port 8000

Open http://<BOARD_IP>:8000. Use --prompts "person|red bus|bicycle" to set startup prompts. Between 1 and 80 non-empty pipe-separated prompts are supported, and each prompt is truncated to 20 CLIP tokens.

Startup arguments

Argument	Default	Description
`--platform`	Required	Target NPU, `rk3576` or `rk3588`.
`--model_path`	Required	Quantized YOLO-World detector.
`--text_model`	`model/clip_text_fp16.rknn`	CLIP text encoder for dynamic prompts.
`--text_features`	`model/coco_text_outp.npy`	Initial `(1,80,512)` COCO features.
`--vocab_path`	`model/clip_vocab.txt`	Offline CLIP BPE vocabulary.
`--class_path`	Built-in COCO labels	Labels matching the initial feature rows.
`--prompts`	COCO 80 classes	Initial pipe-separated prompt phrases.
`--video_path`	None	Looping local MP4; overrides the camera.
`--camera_id`	`1`	Camera index; `-1` enables analysis-only mode.
`--host` / `--port`	`0.0.0.0` / `8000`	Service address and port.

Prompt and prediction APIs

bash

curl -X POST "http://<BOARD_IP>:8000/api/prompts" \
  -H "Content-Type: application/json" \
  -d '{"text":"person|red bus|traffic light"}'

curl -X POST "http://<BOARD_IP>:8000/api/models/yolo_world/predict" \
  -F "file=@bus.jpg" -F "text=person|bus|red vehicle" \
  -F "conf=0.25" -F "iou=0.45"

The text field changes prompts only for that prediction; /api/prompts changes the active stream prompts. Prompt embeddings are cached in memory. Dynamic embeddings fill the first rows of the detector's fixed 80-row text input, and padded rows are ignored during post-processing.

GET /api/prompts reads the active prompts. Health, threshold configuration, MJPEG, video upload, asynchronous analysis, status, list, and download endpoints are also available. The offline BPE vocabulary means the runtime does not download tokenizer assets from Hugging Face.

Model replacement constraints

The detector and text encoder are a matched pipeline. Replacements must retain the detector's 80-row text-feature input, the CLIP embedding width, compatible tokenization, and the output layout expected by the post-processing code.

Build locally

bash

docker build -f docker/rk3576/yolo_world.dockerfile \
  -t rk3576-yolo-world:local src/rk3576_yolo_world

docker build -f docker/rk3588/yolo_world.dockerfile \
  -t rk3588-yolo-world:local src/rk3588_yolo_world

Inputs and Outputs

Input: image, video frame, or camera frame plus up to 80 prompts. Output: prompt-matched boxes and confidence scores.