← Models

CV / Lite Transformer

Lite Transformer

Lite Transformer English-to-Chinese translation accelerated by RKNN on reComputer RK3576 and RK3588.

2 downloads
Precision
RKNN

Choose the device you're using, the set up guide and documentation will update accordingly.

Getting Started

Deploy
sudo docker run --rm --privileged --net=host \
  -e PYTHONUNBUFFERED=1 -e RKNN_LOG_LEVEL=0 \
  -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
  ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-lite_transformer:latest

REST API

Use the REST API to run inference. Copy the commands below.

Curl
curl http://localhost:8080/v1/chat/completions -d '{
  "model": "lite-transformer-rknn",
  "messages": [{"role": "user", "content": "Hello"}]
}'
Python
import requests

resp = requests.post(
    "http://localhost:8080/v1/chat/completions",
    json={"model": "lite-transformer-rknn", "messages": [{"role": "user", "content": "Hello"}]},
)
print(resp.json())

Model Details

Quick Start

1. Install Docker

Run the following commands on the development board to install Docker:

bash
# Download installation script
curl -fsSL https://get.docker.com -o get-docker.sh
# Install using Aliyun mirror source
sudo sh get-docker.sh --mirror Aliyun
# Start Docker and enable auto-start on boot
sudo systemctl enable docker
sudo systemctl start docker

2. Run the Project (One command, Web preview)

This project provides English-to-Chinese translation through a Web browser. The interface displays the translation and the BPE token IDs processed by the RKNN encoder and decoder.

Step A: Pull Images

bash
sudo docker pull ghcr.io/Seeed-Projects/recomputer-rk-cv/rk3588-lite_transformer:latest
sudo docker pull ghcr.io/Seeed-Projects/recomputer-rk-cv/rk3576-lite_transformer:latest

Step B: Run with One Click

For RK3588:

bash
sudo docker run --rm --privileged --net=host \
    -e PYTHONUNBUFFERED=1 \
    -e RKNN_LOG_LEVEL=0 \
    -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
    ghcr.io/seeed-projects/recomputer-rk-cv/rk3588-lite_transformer:latest

Access via: http://<Board_IP>:8000

For RK3576:

bash
sudo docker run --rm --privileged --net=host \
    -e PYTHONUNBUFFERED=1 \
    -e RKNN_LOG_LEVEL=0 \
    -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
    ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-lite_transformer:latest

Access via: http://<Board_IP>:8000

No camera, display, or audio-device mapping is required.

3. Command-Line Inference

bash
python inference.py --platform rk3576 --model_dir model --text "thank you"

Use --platform rk3588 when running the RK3588 source directory.

🔌 API Documentation

The project provides a synchronous REST interface for short English-to-Chinese translation. Interactive OpenAPI documentation is available at http://<Board_IP>:8000/docs.

1. Translation Interface

Endpoint: POST /api/models/lite_transformer/predict

Request Parameters (Multipart/Form-Data):

  • text: Required English source text. The encoded input must not exceed 15 BPE tokens; EOS occupies the remaining fixed input position.

Usage Example:

bash
curl -X POST "http://127.0.0.1:8000/api/models/lite_transformer/predict" \
    -F 'text=thank you'

Response Format (JSON):

json
{
  "success": true,
  "model": "lite_transformer",
  "platform": "rk3576",
  "inference_time_ms": 42.6,
  "result": {
    "input_text": "thank you",
    "translation": "谢谢你",
    "input_tokens": [1234, 5678],
    "output_tokens": [9012, 3456],
    "input_token_count": 2,
    "output_token_count": 2
  }
}

The limit applies to BPE tokens rather than words. A word can produce multiple tokens, and over-length input is rejected instead of silently truncated. This compact model is intended for short phrases and is not comparable to a large general-purpose translation model.

2. System Status and Configuration

Get Current System Status

  • Endpoint: GET /api/health
  • Response: Includes the platform, loaded model files, input type, and runtime capabilities.

Get Current Configuration

  • Endpoint: GET /api/config
  • Response: {} because Lite Transformer has no runtime settings.

Sending non-empty data to POST /api/config is rejected because this model has no hot-swappable runtime parameters.

🛠️ Developer Guide (Production Recommendations)

Code Description

  • web_service.py:
    • Web API: Hosts the browser UI, translation endpoint, health status, and configuration endpoint.
    • Validation: Requires non-empty text and returns input errors as HTTP 400 responses.
  • task_runtime.py:
    • Tokenization: Applies the included BPE data and explicitly validates the 15-token source limit.
    • RKNN Inference: Runs the encoder, then performs autoregressive decoding with the decoder and its state caches.
  • inference.py: Provides one-shot command-line translation.

Model Files

ComponentFile
Encodermodel/encoder.rknn
Decodermodel/decoder.rknn
BPE merge datamodel/bpe_order.txt
Token mapsmodel/cw_token_map_order.txt, model/dict_order.txt
Embeddingsmodel/position_embed.bin, model/token_embed.bin

Modifying Models

  1. Place the converted encoder and decoder in model/ and preserve their filenames.
  2. Keep the tokenizer dictionaries and embedding files aligned with the encoder and decoder conversion.
  3. Verify that the replacement maintains the 15-source-token input and decoder cache contracts.

The encoder and decoder are required stages of one pipeline, not model-size or quantization choices. The upstream project currently provides one compatible bundle for each platform and no separate model-variant parameter.

Build Local Images

Run from the root of reComputer-RK-CV:

bash
docker build -f docker/rk3576/lite_transformer.dockerfile \
    -t rk3576-lite_transformer:local src/rk3576_lite_transformer

docker build -f docker/rk3588/lite_transformer.dockerfile \
    -t rk3588-lite_transformer:local src/rk3588_lite_transformer

Inputs and Outputs

Input: English text of at most 15 source BPE tokens plus EOS. Output: Chinese translation and input/output token details.