CursorCam

Hands-free mouse control via webcam using MediaPipe facial landmark detection. Maps head pitch and yaw angles to screen coordinates in real-time at under 50ms latency. Built for users with motor disabilities or RSI.

The problem

Users with motor disabilities or RSI injuries struggle with conventional mouse/keyboard input. Affordable head-tracking hardware is often inaccessible or overly complex to configure.

The solution

Leveraged MediaPipe facial landmark detection to map head pitch/yaw angles to screen coordinates in real-time via a Flask backend. A lightweight JavaScript frontend handles cursor positioning and gesture-based clicking at sub-50ms latency.

Outcome

Enables fully hands-free mouse control using any standard webcam — no hardware purchase required. Tested with users who reported measurably reduced physical strain during extended sessions.

What I learned

Lighting conditions significantly affect landmark detection accuracy. Would add adaptive brightness preprocessing and per-session sensitivity calibration. Smooth cursor interpolation (lerp) proved essential for usability — raw coordinate mapping was too jittery to be practical.

Stack

Python
Flask
JavaScript
MediaPipe
Computer Vision

Source code