CV / MobileSAM

MobileSAM

Segment objects using a box or foreground/background point prompts in the Web UI or REST API.

16 downloads

Precision

RKNN

Choose the device you're using, the set up guide and documentation will update accordingly.

Platform

Inference Engine

Getting Started

Deploy

sudo docker run --rm --privileged --net=host \
  -e PYTHONUNBUFFERED=1 -e RKNN_LOG_LEVEL=0 \
  --device /dev/dri/renderD129:/dev/dri/renderD129 \
  -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
  ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-mobilesam:latest \
  python web_service.py --platform rk3576 --model_dir /app/model --camera_id -1 --host 0.0.0.0 --port 8000

REST API

Use the REST API to run inference. Copy the commands below.

Curl

curl http://localhost:8080/v1/chat/completions -d '{
  "model": "mobilesam-rknn",
  "messages": [{"role": "user", "content": "Hello"}]
}'

Python

import requests

resp = requests.post(
    "http://localhost:8080/v1/chat/completions",
    json={"model": "mobilesam-rknn", "messages": [{"role": "user", "content": "Hello"}]},
)
print(resp.json())

Model Details

MobileSAM on reComputer RK3576 and RK3588

This deployment packages Mobile Segment Anything from reComputer-RK-CV. The image encoder and fixed two-prompt decoder both run as RKNN models.

Model information

Component	File
Image encoder	`model/mobilesam_encoder.rknn`
Prompt decoder	`model/mobilesam_decoder.rknn`
Input preprocessing	`448 x 448`
Prompts	Box or foreground/background points
Output	Mask, selected-mask index, quality score, and pixel count

Run the service

RK3576

bash

sudo docker run --rm --privileged --net=host \
  -e PYTHONUNBUFFERED=1 -e RKNN_LOG_LEVEL=0 \
  --device /dev/dri/renderD129:/dev/dri/renderD129 \
  -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
  ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-mobilesam:latest \
  python web_service.py --platform rk3576 --model_dir /app/model \
  --camera_id -1 --host 0.0.0.0 --port 8000

RK3588

bash

sudo docker run --rm --privileged --net=host \
  -e PYTHONUNBUFFERED=1 -e RKNN_LOG_LEVEL=0 \
  --device /dev/dri/renderD129:/dev/dri/renderD129 \
  -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
  ghcr.io/seeed-projects/recomputer-rk-cv/rk3588-mobilesam:latest \
  python web_service.py --platform rk3588 --model_dir /app/model \
  --camera_id -1 --host 0.0.0.0 --port 8000

Open http://<BOARD_IP>:8000 to upload an image and draw a box or add foreground/background points. The current prompt is shared by image, camera, local-video, and uploaded-video inference until it is changed.

Input modes and startup arguments

Argument	Default	Description
`--platform`	Required	`rk3576` or `rk3588`.
`--model_dir`	`model`	Encoder, decoder, and warm-up image directory.
`--camera_id`	`-1`	Camera index; `-1` enables uploads only.
`--video`, `--video_path`	None	Looping local video; overrides the camera.
`--host` / `--port`	`0.0.0.0` / `8000`	Service address and port.

The Web UI supports a dragged box, positive point, negative point, and a full-image prompt. Updating the prompt affects subsequent stream and uploaded video frames without restarting the container.

REST API

Endpoint: POST /api/models/mobilesam/predict

The multipart request accepts file, point_coords, and point_labels. point_coords must contain exactly two source-image coordinates. SAM prompt labels are 0 for a negative point, 1 for a positive point, 2 for the top-left box corner, and 3 for the bottom-right box corner.

bash

curl -X POST "http://<BOARD_IP>:8000/api/models/mobilesam/predict" \
  -F "file=@picture.jpg" \
  -F 'point_coords=[[190,70],[460,280]]' \
  -F 'point_labels=[2,3]'

The runtime scales prompt coordinates to the encoder input, runs the encoder and decoder, selects the requested or best-scoring mask, and restores it to the source image. Health, configuration, MJPEG, and asynchronous-video interfaces are also available.

The response contains iou_scores, selected_mask, mask_pixels, and the effective prompt. GET /api/config returns the persistent stream prompt; POST /api/config accepts point_coords and point_labels, or both as null to restore full-image mode. Generic threshold and topk values do not change mask selection.

Uploaded videos use the persistent prompt active when analysis starts. The standard /api/video/* endpoints provide upload, analysis, progress, list, and download operations.

The encoder and decoder are pipeline stages, not model-size alternatives. Both files must remain compatible when replacing either model. The encoder preserves aspect ratio, resizes the long side to 448, and pads to 448 x 448; the fixed decoder contract requires exactly two prompt coordinates and labels.

Build locally

bash

docker build -f docker/rk3576/mobilesam.dockerfile \
  -t rk3576-mobilesam:local src/rk3576_mobilesam

docker build -f docker/rk3588/mobilesam.dockerfile \
  -t rk3588-mobilesam:local src/rk3588_mobilesam

Inputs and Outputs

Input: image plus box or point prompts. Output: segmentation mask, mask score, and overlay.