William Zhang2026-09-27

9-Filter Dual-Hand Pinch Demo with MediaPipe Hand Landmarker

Real-time detection of 21 hand key points and pinch state for both hands on CPU (XNNPACK-accelerated) using the MediaPipe Tasks Hand Landmarker model. "Both hands pinching simultaneously" maps directly to a quadrilateral ROI, and 9 image filters are applied live to the pixels inside the ROI.

CVHand TrackingMediaPipeGithub

Visual filter showcase based on reComputer-RK / Docker / RKNN / YOLO

https://github.com/doublelf/filter_game

Project Overview

This project consolidates 9 filters into a single FILTER_LIST dictionary for easy extension—yielding a 191-line, filter-only studio: gesture.py detects both-hand pinches → filter_game.FilterGame maintains the quadrilateral vertices → filters.apply_filter processes the ROI pixels.

Quick Start

1. Clone and enter the project directory

bash
git clone https://github.com/doublelf/filter_game
cd filter_game

2. Install dependencies

bash
pip install -r requirements.txt

requirements.txt pins four dependencies:

text
opencv-python>=4.8.0
mediapipe>=0.10.14
numpy>=1.26.0
Pillow>=10.4.0

Of the 9 filters, EMBOSS / CONTOUR / POSTERIZE are provided by Pillow; INVERT / NEON / GRAY / SEPIA / HUE_SHIFT / PIXELATE are implemented with OpenCV + NumPy and still work without Pillow (returning the original frame when Pillow is missing, see filters.py:36 _pil_filter_safe).

3. Launch the game

bash
python main.py                  # default camera 0, horizontal flip enabled
python main.py --cam 1          # use the 2nd camera
python main.py --no-flip        # disable horizontal flip
python main.py --no-landmarks   # disable the key-point overlay (enabled by default)
python main.py --hide-trails    # disable gesture trails (enabled by default)

After launching you will first see the "Filter Studio" menu screen; press any key to enter the game.

4. Controls

ActionPlayer InputProgram Feedback
Single-hand pinchAny hand starts pinchingGolden spread-ring at the index fingertip
Both hands pinchBoth hands pinch at the same timeSpread-ring at the midpoint between the two anchors + Toast CONNECTED
Hold pinch + moveDrag both handsQuadrilateral (anchor_l, anchor_r, current_l, current_r) with the current filter applied to its inner pixels
Release either handRelease one handQuadrilateral disappears, returns to IDLE
Horizontal swipe to switch filterSwipe across the screen with any non-pinching handSwipeIndicator side gradient band + Toast Filter N/9 NAME
Keyboard filter switch1–9 direct jump / [ ] or , . next/prevToast Filter N/9 NAME
AnytimeM returns to menuCamera keeps capturing, only the game loop is closed
AnytimeESC quitsReleases the camera, closes MediaPipe, destroys the window

Running Without a Camera

recomputer-rk3576-devkit ships without a USB camera connected by default. This project bundles three utilities so it can run end-to-end even without a camera:

tools/verify_all.py — 8-step verification suite

bash
cd /home/seeed/hand/filter_game
python3 -m tools.verify_all

Prints each step:

  • Module imports (gesture / filter_game / filters / overlay / utils)
  • utils geometry + text helpers
  • All 9 filters output correct shape and dtype
  • FilterGame state machine + SwipeTracker horizontal-swipe detection
  • Real MediaPipe Hand Landmarker loaded on aarch64 (XNNPACK delegate, ~110 ms warm-up)
  • HandOverlay.add_event + rendering (both_connected / filter_switch / swipe)
  • 120-frame synthesized both-hand pipeline finishes, and FilterGame has reached BOTH_PINCH at least once

tools/run_synthetic.py — Offline rendering to PNG

bash
python3 -m tools.run_synthetic --frames 200 --out /tmp/filter_demo
ls /tmp/filter_demo | wc -l      # → 200

Drives the full FilterGame + HandOverlay pipeline and prints:

Code
events total=N by_type={'both_connected': 1, ...}
BOTH_PINCH frames: 1   DRAG frames: 146

Writes each frame to out/filter_XXXX.png. In a camera-less environment, stitch the PNGs into an mp4 with ffmpeg:

bash
ffmpeg -framerate 30 -i /tmp/filter_demo/filter_%04d.png -c:v libx264 -pix_fmt yuv420p filter.mp4

tools/synthetic_source.py + tools/fake_tracker.py

A synthesized video source and a synthesized HandTracker that are interface-equivalent to cv2.VideoCapture / gesture.HandTracker. To switch to a real camera later, replace just these two lines:

python
cap = cv2.VideoCapture(args.cam)               # replace SyntheticHandSource
tracker = HandTracker(num_hands=2)              # replace SyntheticHandTracker

Then tools/run_synthetic.py upgrades into a real-camera driver script.