← Models

VLM / Qwen3.5

Qwen3.5 4B

This published VLM image combines the Qwen3.5 4B language model with its RKNN vision encoder and an OpenAI-compatible multimodal API.

23 downloads
Size
3.81GB

Choose the device you're using, the set up guide and documentation will update accordingly.

Getting Started

Deploy
sudo docker run --rm -d \
  --name recomputer-rk-vlm \
  --privileged \
  -p 8001:8001 \
  -v /dev:/dev \
  -e MODEL_KIND=vlm \
  -e LOG_LEVEL=warning \
  ghcr.io/seeed-projects/recomputer-rk-llm/vlm/qwen3.5-4b:rk3576-w4a16-g128

Model Details

VLM deployment on reComputer RK

This shared guide applies to every VLM listed in the catalog, including Qwen3.5. Each published VLM image contains a matching RKLLM language model and RKNN vision encoder for the selected board and quantization.

Published image format

text
ghcr.io/seeed-projects/recomputer-rk-llm/vlm/<model-id>:<platform>-<quantization>

W4A16-G128 images are generally RK3576-only. RK3588 VLM records use W8A8.

Requirements

  • reComputer RK3576 or RK3588 running a 64-bit Linux image
  • Docker and Docker Buildx installed
  • NPU access through /dev
  • Port 8001 available on the board

VLM images use the OpenAI-compatible API and do not provide terminal interactive chat.

Run a published VLM

Replace <model-id> and <quantization> with the selected catalog record.

RK3576

bash
sudo docker run --rm -d \
  --name recomputer-rk-vlm \
  --privileged \
  -p 8001:8001 \
  -v /dev:/dev \
  -e MODEL_KIND=vlm \
  -e LOG_LEVEL=warning \
  ghcr.io/seeed-projects/recomputer-rk-llm/vlm/<model-id>:rk3576-<quantization>

RK3588

bash
sudo docker run --rm -d \
  --name recomputer-rk-vlm \
  --privileged \
  -p 8001:8001 \
  -v /dev:/dev \
  -e MODEL_KIND=vlm \
  -e LOG_LEVEL=warning \
  ghcr.io/seeed-projects/recomputer-rk-llm/vlm/<model-id>:rk3588-w8a8

Health and API documentation

bash
curl http://localhost:8001/health
curl http://localhost:8001/docs
curl http://localhost:8001/redoc

OpenAI multimodal request

The server accepts one image per request. Use a public HTTP(S) URL or a base64 data URL.

bash
curl -X POST http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rkllm-vision",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe this image in detail."},
        {"type": "image_url", "image_url": {"url": "https://example.com/your-image.jpg"}}
      ]
    }],
    "stream": false
  }'

For a local image, put a data URL in image_url.url:

text
data:image/jpeg;base64,<base64-encoded-image>

Streaming multimodal request

bash
curl -N -X POST http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rkllm-vision",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "List the objects visible in this image."},
        {"type": "image_url", "image_url": {"url": "https://example.com/your-image.jpg"}}
      ]
    }],
    "stream": true
  }'

Python client:

python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8001/v1", api_key="dummy-key")
response = client.chat.completions.create(
    model="rkllm-vision",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image."},
            {"type": "image_url", "image_url": {"url": "https://example.com/your-image.jpg"}},
        ],
    }],
    max_tokens=128,
)
print(response.choices[0].message.content)

Cherry Studio

Configure a custom OpenAI-compatible provider:

  • API host: http://<board-ip>:8001/v1
  • API key: any value, such as rkllm-local
  • Model: rkllm-vision

Attach an image after selecting the provider.

Troubleshooting

  • Do not mix language-model and vision-encoder artifacts from different boards or tags.
  • If the server reports a missing vision model, confirm MODEL_KIND=vlm and the published image tag.
  • Inspect startup output with sudo docker logs recomputer-rk-vlm.

Inputs and Outputs

Input: an image and text prompt. Output: image-grounded generated text through an OpenAI-compatible API.