Qwen2.5-VL 3B Instruct

VLM deployment on reComputer RK

This shared guide applies to every VLM listed in the catalog, including Qwen3.5. Each published VLM image contains a matching RKLLM language model and RKNN vision encoder for the selected board and quantization.

The current upstream environment uses RKLLM v1.3.1 and RKNN runtime / Toolkit Lite 2 v2.5.0. New or updated model definitions must be built and published before their tags can be pulled. Build the environment image first, then rebuild the selected model images; see the upstream repository and VLM deployment guide.

Model profiles and Gemma 4 preview

Model images supply their matching runtime settings. Qwen2.5-VL and Qwen3-VL use VLM_IMAGE_SIZE=448. Gemma 4 uses VLM_MODEL_FAMILY=gemma4, VLM_IMAGE_SIZE=480, and RKLLM_EMBED_FLASH=1, and includes an audio encoder. For custom artifacts, follow the matching upstream model definition rather than mixing model profiles or encoder files.

Gemma 4 can accept a base64 WAV input_audio content part through the same chat completions endpoint. Custom Gemma 4 deployments must load the matching audio encoder through AUDIO_MODEL_FILE or AUDIO_MODEL_PATH; see the upstream VLM deployment guide for configuration.

Published image format

ghcr.io/seeed-projects/recomputer-rk-llm/vlm/<model-id>:<platform>-<quantization>

W4A16-G128 images are generally RK3576-only. RK3588 VLM records use W8A8.

Requirements

• reComputer RK3576 or RK3588 running a 64-bit Linux image
• Docker and Docker Buildx installed
• NPU access through /dev
• Port 8001 available on the board

VLM images use the OpenAI-compatible API and do not provide terminal interactive chat.

Run a published VLM

Replace <model-id> and <quantization> with the selected catalog record.

RK3576

sudo docker run --rm -d \
  --name recomputer-rk-vlm \
  --privileged \
  -p 8001:8001 \
  -v /dev:/dev \
  -e MODEL_KIND=vlm \
  -e LOG_LEVEL=warning \
  ghcr.io/seeed-projects/recomputer-rk-llm/vlm/<model-id>:rk3576-<quantization>

RK3588

sudo docker run --rm -d \
  --name recomputer-rk-vlm \
  --privileged \
  -p 8001:8001 \
  -v /dev:/dev \
  -e MODEL_KIND=vlm \
  -e LOG_LEVEL=warning \
  ghcr.io/seeed-projects/recomputer-rk-llm/vlm/<model-id>:rk3588-w8a8

Health and API documentation

curl http://localhost:8001/health
curl http://localhost:8001/docs
curl http://localhost:8001/redoc

OpenAI multimodal request

The server accepts one image per request. Use a public HTTP(S) URL or a base64 data URL.

curl -X POST http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rkllm-vision",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe this image in detail."},
        {"type": "image_url", "image_url": {"url": "https://example.com/your-image.jpg"}}
      ]
    }],
    "stream": false
  }'

For a local image, put a data URL in image_url.url:

data:image/jpeg;base64,<base64-encoded-image>

Streaming multimodal request

curl -N -X POST http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rkllm-vision",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "List the objects visible in this image."},
        {"type": "image_url", "image_url": {"url": "https://example.com/your-image.jpg"}}
      ]
    }],
    "stream": true
  }'

Python client:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8001/v1", api_key="dummy-key")
response = client.chat.completions.create(
    model="rkllm-vision",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image."},
            {"type": "image_url", "image_url": {"url": "https://example.com/your-image.jpg"}},
        ],
    }],
    max_tokens=128,
)
print(response.choices[0].message.content)

Cherry Studio

Configure a custom OpenAI-compatible provider:

• API host: http://<board-ip>:8001/v1
• API key: any value, such as rkllm-local
• Model: rkllm-vision

Attach an image after selecting the provider.

Troubleshooting

• Do not mix language-model and vision-encoder artifacts from different boards or tags.
• If the server reports a missing vision model, confirm MODEL_KIND=vlm and the published image tag.
• Inspect startup output with sudo docker logs recomputer-rk-vlm.