Lite Transformer

Quick Start

1. Install Docker

Run the following commands on the development board to install Docker:

# Download installation script
curl -fsSL https://get.docker.com -o get-docker.sh
# Install using Aliyun mirror source
sudo sh get-docker.sh --mirror Aliyun
# Start Docker and enable auto-start on boot
sudo systemctl enable docker
sudo systemctl start docker

2. Run the Project (One command, Web preview)

This project provides English-to-Chinese translation through a Web browser. The interface displays the translation and the BPE token IDs processed by the RKNN encoder and decoder.

Step A: Pull Images

sudo docker pull ghcr.io/Seeed-Projects/recomputer-rk-cv/rk3588-lite_transformer:latest
sudo docker pull ghcr.io/Seeed-Projects/recomputer-rk-cv/rk3576-lite_transformer:latest

Step B: Run with One Click

For RK3588:

sudo docker run --rm --privileged --net=host \
    -e PYTHONUNBUFFERED=1 \
    -e RKNN_LOG_LEVEL=0 \
    -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
    ghcr.io/seeed-projects/recomputer-rk-cv/rk3588-lite_transformer:latest

Access via: http://<Board_IP>:8000

For RK3576:

sudo docker run --rm --privileged --net=host \
    -e PYTHONUNBUFFERED=1 \
    -e RKNN_LOG_LEVEL=0 \
    -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
    ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-lite_transformer:latest

Access via: http://<Board_IP>:8000

No camera, display, or audio-device mapping is required.

3. Command-Line Inference

python inference.py --platform rk3576 --model_dir model --text "thank you"

Use --platform rk3588 when running the RK3588 source directory.

🔌 API Documentation

The project provides a synchronous REST interface for short English-to-Chinese translation. Interactive OpenAPI documentation is available at http://<Board_IP>:8000/docs.

1. Translation Interface

Endpoint: POST /api/models/lite_transformer/predict

Request Parameters (Multipart/Form-Data):

• text: Required English source text. The encoded input must not exceed 15 BPE tokens; EOS occupies the remaining fixed input position.

Usage Example:

curl -X POST "http://127.0.0.1:8000/api/models/lite_transformer/predict" \
    -F 'text=thank you'

Response Format (JSON):

{
  "success": true,
  "model": "lite_transformer",
  "platform": "rk3576",
  "inference_time_ms": 42.6,
  "result": {
    "input_text": "thank you",
    "translation": "谢谢你",
    "input_tokens": [1234, 5678],
    "output_tokens": [9012, 3456],
    "input_token_count": 2,
    "output_token_count": 2
  }
}

The limit applies to BPE tokens rather than words. A word can produce multiple tokens, and over-length input is rejected instead of silently truncated. This compact model is intended for short phrases and is not comparable to a large general-purpose translation model.

2. System Status and Configuration

Get Current System Status

• Endpoint: GET /api/health
• Response: Includes the platform, loaded model files, input type, and runtime capabilities.

Get Current Configuration

• Endpoint: GET /api/config
• Response: {} because Lite Transformer has no runtime settings.

Sending non-empty data to POST /api/config is rejected because this model has no hot-swappable runtime parameters.

🛠️ Developer Guide (Production Recommendations)

Code Description

• web_service.py:
  • Web API: Hosts the browser UI, translation endpoint, health status, and configuration endpoint.
  • Validation: Requires non-empty text and returns input errors as HTTP 400 responses.
• task_runtime.py:
  • Tokenization: Applies the included BPE data and explicitly validates the 15-token source limit.
  • RKNN Inference: Runs the encoder, then performs autoregressive decoding with the decoder and its state caches.
• inference.py: Provides one-shot command-line translation.

Model Files

Modifying Models

1. Place the converted encoder and decoder in model/ and preserve their filenames.
2. Keep the tokenizer dictionaries and embedding files aligned with the encoder and decoder conversion.
3. Verify that the replacement maintains the 15-source-token input and decoder cache contracts.

The encoder and decoder are required stages of one pipeline, not model-size or quantization choices. The upstream project currently provides one compatible bundle for each platform and no separate model-variant parameter.

Build Local Images

Run from the root of reComputer-RK-CV:

docker build -f docker/rk3576/lite_transformer.dockerfile \
    -t rk3576-lite_transformer:local src/rk3576_lite_transformer

docker build -f docker/rk3588/lite_transformer.dockerfile \
    -t rk3588-lite_transformer:local src/rk3588_lite_transformer