YAMNet

Quick Start

1. Install Docker

Run the following commands on the development board to install Docker:

# Download installation script
curl -fsSL https://get.docker.com -o get-docker.sh
# Install using Aliyun mirror source
sudo sh get-docker.sh --mirror Aliyun
# Start Docker and enable auto-start on boot
sudo systemctl enable docker
sudo systemctl start docker

2. Run the Project (One command, dual-mode preview)

This project classifies audio across 521 AudioSet classes. The Web interface accepts uploaded audio or browser microphone recordings and displays both overall Top-K predictions and a sliding-window event timeline.

Step A: Pull Images

sudo docker pull ghcr.io/Seeed-Projects/recomputer-rk-cv/rk3588-yamnet:latest
sudo docker pull ghcr.io/Seeed-Projects/recomputer-rk-cv/rk3576-yamnet:latest

Step B: Run with One Click

For RK3588:

sudo docker run --rm --privileged --net=host \
    -e PYTHONUNBUFFERED=1 \
    -e RKNN_LOG_LEVEL=0 \
    -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
    ghcr.io/seeed-projects/recomputer-rk-cv/rk3588-yamnet:latest

Access via: http://<Board_IP>:8000

For RK3576:

sudo docker run --rm --privileged --net=host \
    -e PYTHONUNBUFFERED=1 \
    -e RKNN_LOG_LEVEL=0 \
    -v /proc/device-tree/compatible:/proc/device-tree/compatible:ro \
    ghcr.io/seeed-projects/recomputer-rk-cv/rk3576-yamnet:latest

Access via: http://<Board_IP>:8000

Browser recording is uploaded to the service, so host audio-device mapping is not required.

3. Command-Line Inference

python inference.py --platform rk3576 --model_dir model \
    --file samples/test.wav --topk 5 --threshold 0.1 --hop_seconds 1

Use --platform rk3588 when running the RK3588 source directory.

🔌 API Documentation

The REST API converts audio to mono 16 kHz and analyzes fixed 3-second windows. Input duration is limited to 300 seconds.

1. Audio Event Classification Interface

Endpoint: POST /api/models/yamnet/predict

Request Parameters (Multipart/Form-Data):

• file: Required audio file. The upload limit is 64 MB.
• topk: Optional number of highest-scoring classes, from 1 to 20. The default is 5.
• threshold: Optional minimum event score, from 0 to 1. The default is 0.1.
• hop_seconds: Optional distance between successive 3-second windows, from 0.25 to 3 seconds. The default is 1 second.

Usage Example:

curl -X POST "http://127.0.0.1:8000/api/models/yamnet/predict" \
    -F 'file=@samples/test.wav' \
    -F 'topk=5' \
    -F 'threshold=0.1' \
    -F 'hop_seconds=1'

Response Format (JSON):

{
  "success": true,
  "model": "yamnet",
  "platform": "rk3576",
  "inference_time_ms": 76.1,
  "result": {
    "duration_seconds": 4.5,
    "sample_rate": 16000,
    "window_seconds": 3.0,
    "hop_seconds": 1.0,
    "top_class": {"id": 0, "label": "Speech", "score": 0.83},
    "predictions": [],
    "segments": [],
    "events": []
  }
}

predictions contains overall scores, segments contains per-window results, and events merges consecutive windows whose top class is the same.

2. System Configuration Interface

Get Current Configuration

• Endpoint: GET /api/config
• Default Response: {"topk": 5, "threshold": 0.1, "hop_seconds": 1.0}

Update Configuration

curl -X POST "http://127.0.0.1:8000/api/config" \
    -H 'Content-Type: application/json' \
    -d '{"topk": 10, "threshold": 0.2, "hop_seconds": 0.5}'

Use GET /api/health to inspect the active platform, loaded model, and runtime capabilities. Interactive OpenAPI documentation is at /docs.

🛠️ Developer Guide (Production Recommendations)

Code Description

• web_service.py:
  • Web API: Handles uploaded files, browser recordings, runtime configuration, and classification responses.
  • Validation: Enforces upload, duration, and parameter limits.
• task_runtime.py:
  • Audio Processing: Converts input to mono 16 kHz and schedules fixed 3-second windows.
  • Classification: Runs yamnet.rknn, ranks 521 labels, and applies the Top-K and threshold settings.
  • Timeline: Produces per-window segments and merges consecutive events.
• inference.py: Provides one-shot command-line classification.

Model Files

Modifying Models

1. Place the converted model at model/yamnet.rknn and the aligned 521-label map at model/yamnet_class_map.txt.
2. Preserve the mono 16 kHz, fixed 3-second input and output class ordering.
3. Test short clips, long sliding-window input, and each parameter boundary.

The upstream project currently provides one RKNN model per platform. topk, threshold, and hop_seconds control post-processing and window scheduling rather than model size or quantization.

Build Local Images

Run from the root of reComputer-RK-CV:

docker build -f docker/rk3576/yamnet.dockerfile \
    -t rk3576-yamnet:local src/rk3576_yamnet

docker build -f docker/rk3588/yamnet.dockerfile \
    -t rk3588-yamnet:local src/rk3588_yamnet