← Models

VLM / Qwen3.5

Qwen3.5 4B

An RKLLM/RKNN-optimized Qwen3.5 4B vision-language model for image understanding and text generation on reComputer RK boards.

79 downloads
Size
3.81GB

Choose the device you're using, the set up guide and documentation will update accordingly.

Getting Started

Deploy
sudo docker run --rm -d \
  --name recomputer-rk-vlm \
  --privileged \
  -p 8001:8001 \
  -v /dev:/dev \
  -e MODEL_KIND=vlm \
  -e LOG_LEVEL=warning \
  ghcr.io/seeed-projects/recomputer-rk-llm/vlm/qwen3.5-4b:rk3576-w4a16-g128

Model Details

VLM deployment on reComputer RK

This shared guide applies to every VLM listed in the catalog, including Qwen3.5. Each published VLM image contains a matching RKLLM language model and RKNN vision encoder for the selected board and quantization.

Published image format

text
ghcr.io/seeed-projects/recomputer-rk-llm/vlm/<model-id>:<platform>-<quantization>

W4A16-G128 images are generally RK3576-only. RK3588 VLM records use W8A8.

Requirements

  • reComputer RK3576 or RK3588 running a 64-bit Linux image
  • Docker and Docker Buildx installed
  • NPU access through /dev
  • Port 8001 available on the board

VLM images use the OpenAI-compatible API and do not provide terminal interactive chat.

Run a published VLM

Replace <model-id> and <quantization> with the selected catalog record.

RK3576

bash
sudo docker run --rm -d \
  --name recomputer-rk-vlm \
  --privileged \
  -p 8001:8001 \
  -v /dev:/dev \
  -e MODEL_KIND=vlm \
  -e LOG_LEVEL=warning \
  ghcr.io/seeed-projects/recomputer-rk-llm/vlm/<model-id>:rk3576-<quantization>

RK3588

bash
sudo docker run --rm -d \
  --name recomputer-rk-vlm \
  --privileged \
  -p 8001:8001 \
  -v /dev:/dev \
  -e MODEL_KIND=vlm \
  -e LOG_LEVEL=warning \
  ghcr.io/seeed-projects/recomputer-rk-llm/vlm/<model-id>:rk3588-w8a8

Health and API documentation

bash
curl http://localhost:8001/health
curl http://localhost:8001/docs
curl http://localhost:8001/redoc

OpenAI multimodal request

The server accepts one image per request. Use a public HTTP(S) URL or a base64 data URL.

bash
curl -X POST http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rkllm-vision",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe this image in detail."},
        {"type": "image_url", "image_url": {"url": "https://example.com/your-image.jpg"}}
      ]
    }],
    "stream": false
  }'

For a local image, put a data URL in image_url.url:

text
data:image/jpeg;base64,<base64-encoded-image>

Streaming multimodal request

bash
curl -N -X POST http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rkllm-vision",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "List the objects visible in this image."},
        {"type": "image_url", "image_url": {"url": "https://example.com/your-image.jpg"}}
      ]
    }],
    "stream": true
  }'

Python client:

python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8001/v1", api_key="dummy-key")
response = client.chat.completions.create(
    model="rkllm-vision",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image."},
            {"type": "image_url", "image_url": {"url": "https://example.com/your-image.jpg"}},
        ],
    }],
    max_tokens=128,
)
print(response.choices[0].message.content)

Cherry Studio

Configure a custom OpenAI-compatible provider:

  • API host: http://<board-ip>:8001/v1
  • API key: any value, such as rkllm-local
  • Model: rkllm-vision

Attach an image after selecting the provider.

Troubleshooting

  • Do not mix language-model and vision-encoder artifacts from different boards or tags.
  • If the server reports a missing vision model, confirm MODEL_KIND=vlm and the published image tag.
  • Inspect startup output with sudo docker logs recomputer-rk-vlm.

Inputs and Outputs

Input: an image and text prompt. Output: image-grounded generated text through an OpenAI-compatible API.