MMS-TTS

Quick Start

1. Install Docker

Run the following commands on the development board to install Docker:

# Download installation script
curl -fsSL https://get.docker.com -o get-docker.sh
# Install using Aliyun mirror source
sudo sh get-docker.sh --mirror Aliyun
# Start Docker and enable auto-start on boot
sudo systemctl enable docker
sudo systemctl start docker

2. Run the Project (One command, Web preview)

This project provides English speech synthesis with facebook/mms-tts-eng. The Web interface accepts text, controls speaking rate, and provides WAV playback and download.

Step A: Pull Images

sudo docker pull ghcr.io/Seeed-Projects/recomputer-rk-cv/rk3588-mms_tts:latest
sudo docker pull ghcr.io/Seeed-Projects/recomputer-rk-cv/rk3576-mms_tts:latest

Step B: Run with One Click

For RK3588:

sudo docker run --rm --privileged --net=host \
    -e PYTHONUNBUFFERED=1 \
    -e RKNN_LOG_LEVEL=0 \
    -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
    ghcr.io/seeed-projects/recomputer-rk-cv/rk3588-mms_tts:latest

Access via: http://<Board_IP>:8000

For RK3576:

sudo docker run --rm --privileged --net=host \
    -e PYTHONUNBUFFERED=1 \
    -e RKNN_LOG_LEVEL=0 \
    -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
    ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-mms_tts:latest

Access via: http://<Board_IP>:8000

3. Command-Line Inference

python inference.py --platform rk3576 --model_dir model \
    --text 'hello from the edge' \
    --speaking_rate 0.8 \
    --output output.wav

Use --platform rk3588 when running the RK3588 source directory.

🔌 API Documentation

The REST API generates English speech synchronously and stores the newest 20 generated WAV files for temporary download.

1. Speech Synthesis Interface

Endpoint: POST /api/models/mms_tts/predict

Request Parameters (Multipart/Form-Data):

• text: Required English text containing only supported characters, with a maximum of 99 characters.
• speaking_rate: Optional value from 0.6 to 1.4. The default is 1.0; values below 1 produce slower speech.

Usage Example:

curl -X POST "http://127.0.0.1:8000/api/models/mms_tts/predict" \
    -F 'text=hello from the edge' \
    -F 'speaking_rate=0.8'

Response Format (JSON):

{
  "success": true,
  "model": "mms_tts",
  "platform": "rk3576",
  "inference_time_ms": 224.7,
  "result": {
    "text": "hello from the edge",
    "sample_rate": 16000,
    "samples": 35840,
    "duration_seconds": 2.24,
    "speaking_rate": 0.8,
    "duration_clipped": false,
    "warnings": [],
    "audio_url": "/api/audio/<audio_filename>",
    "audio_filename": "<audio_filename>"
  }
}

The speaking-rate parameter scales phoneme durations before decoding, so it does not post-resample the WAV and preserves pitch. If slow speech or long text reaches the decoder duration limit, duration_clipped is true and the response includes a warning.

2. Download Generated Audio

Endpoint: GET /api/audio/{audio_filename}

curl -o speech.wav \
    "http://127.0.0.1:8000/api/audio/<audio_filename>"

Use the exact audio_filename returned by the prediction interface.

3. System Configuration Interface

Get Current Configuration

• Endpoint: GET /api/config
• Default Response: {"speaking_rate": 1.0}

Update Configuration

curl -X POST "http://127.0.0.1:8000/api/config" \
    -H 'Content-Type: application/json' \
    -d '{"speaking_rate": 1.2}'

Use GET /api/health to inspect the active platform, loaded encoder and decoder, and supported input capabilities. OpenAPI documentation is at /docs.

🛠️ Developer Guide (Production Recommendations)

Code Description

• web_service.py:
  • Web API: Handles synthesis requests, configuration, generated-audio storage, playback, and download.
  • Output Retention: Creates UUID-based WAV filenames and retains the newest 20 generated files.
• task_runtime.py:
  • Text Processing: Validates supported English characters and converts them to the fixed model input.
  • Duration Control: Scales phoneme durations with speaking_rate and reports decoder clipping.
  • RKNN Inference: Runs the encoder and decoder and writes mono 16 kHz WAV output.
• inference.py: Provides one-shot synthesis to a local WAV file.

Model Files

Modifying Models

1. Place compatible encoder and decoder models in model/ and preserve their filenames.
2. Keep the pair aligned with the expected vocabulary, duration tensor, and mono 16 kHz output contract.
3. Test the 0.6–1.4 speaking-rate range and the decoder duration limit.

The encoder and decoder are two stages of one synthesis pipeline rather than alternative sizes. The upstream project currently provides one pair per platform and no separate quantization selector.

Build Local Images

Run from the root of reComputer-RK-CV:

docker build -f docker/rk3576/mms_tts.dockerfile \
    -t rk3576-mms_tts:local src/rk3576_mms_tts

docker build -f docker/rk3588/mms_tts.dockerfile \
    -t rk3588-mms_tts:local src/rk3588_mms_tts