MotionJoy turns body movement into keyboard input. Lean left or right, jump, or crouch in front of your webcam and the game you are playing receives the matching arrow key. It needs no extra hardware, runs fully on your computer, and uses ready-made presets for games.
Status: prototype. MotionJoy is an early student project (JDH Group, Institut Teknologi Indonesia). It works end to end, but gesture accuracy has not been formally measured and some gestures (especially crouch) still need tuning.
- Webcam only. No sensors, no controller. Pose tracking runs locally with MediaPipe.
- Game presets. Subway Surfers and Chrome Dino included. Add your own with a small JSON file (docs).
- Floating panel that does not steal focus. Always on top of the game; resize it, hide the camera, or move it anywhere.
- Sensitivity you can tune live. Low / Normal / High / Custom, with sliders for Lean, Jump and Crouch while you play.
- Automatic calibration. Stand in your play position and the app calibrates after a short countdown. One click for a new player.
- Lighting check. Warns when the room is too dark, too bright, or you are backlit.
- Private by design. No video is recorded or sent anywhere. The app works offline (details).
- Open the Releases page and download one of:
MotionJoy-Setup-<version>.exe: installerMotionJoy-Portable-<version>.exe: runs without installing
- Windows may show "Windows protected your PC" because the app is not code-signed yet. Click More info, then Run anyway.
You can compare the file with
SHA256SUMS.txtfrom the same release. - Start the game you want to play first, then start MotionJoy.
Requirements: Windows 10 or 11 (64-bit) and a webcam. Run MotionJoy and the game at the same privilege level (both normal, not Administrator).
- Setup: stand 1.5 to 2 m from the camera so your head to hips is visible. Check the lighting badge.
- Choose game: pick a game. Press Test keys (5 s), then click the game window to confirm it receives the keys.
- Start playing: the panel shrinks and floats above the game. Click the game window. After the countdown, MotionJoy calibrates (stand still for 3 seconds).
- Play: lean, jump, crouch. Use the icons on the panel:
| Icon | What it does |
|---|---|
| Rotate | Recalibrate / new player |
| Pause | Pause or resume key sending |
| Sliders | Sensitivity (Low, Normal, High, Custom) |
| Eye | Hide or show the camera preview |
| Gear | Settings, session log, about |
| X | Quit |
Drag the panel header to move it and the bottom-right corner to resize it.
| Gesture | Default key (Subway Surfers) |
|---|---|
| Lean left / right | Left / Right arrow |
| Jump | Up arrow |
| Crouch | Down arrow |
git clone https://github.com/MuhammadZaidan1/motionjoy-controller-desktop.git
cd motionjoy-controller-desktop
npm install
npm startKey sending only works on Windows. The engine and tests run on any OS: npm test.
On a Windows machine:
npm install
npm run distOutput goes to dist/: an installer (MotionJoy-Setup-*.exe) and a portable exe (MotionJoy-Portable-*.exe).
If the build fails with a symbolic-link error, see Troubleshooting.
Releases are built by GitHub Actions. Update version in package.json, commit, then:
git tag v0.9.0-prototype
git push origin v0.9.0-prototypeThe workflow builds on Windows, runs the tests, and publishes a Release with the executables and checksums.
The tag must match package.json (v + version).
src/
main.js, preload.js, keys.js Electron main process, bridge, key injection (Windows SendInput)
engine/ Pure logic: metrics, gestures, presets, sensitivity, lighting
ui/ Interface (HTML, CSS, ES modules) and design system
vendor/ Bundled MediaPipe runtime and pose models (offline)
presets/ Game presets (JSON)
test/ Unit tests for the engine
docs/ Architecture, presets, privacy, troubleshooting, booth guide
build/ App icon
.github/ CI, release workflow, issue and PR templates
More detail in docs/ARCHITECTURE.md.
- Prototype: not formally evaluated; do not treat it as an accuracy claim.
- Crouch detection depends on camera angle and framing and still needs tuning.
- Windows only. Games with anti-cheat or elevated rights may ignore injected keys.
- Keys are sent to whichever window has focus. Click the game window before playing.
- The executable is not code-signed, so SmartScreen shows a warning.
- Measure gesture reliability with real players and tune defaults.
- More game presets and an in-app preset editor.
- Code signing and auto-update.
- Optional gamepad emulation.
Issues and pull requests are welcome. See CONTRIBUTING.md.
JDH Group, Institut Teknologi Indonesia: Muhammad Zaidan Fadlurrahman, Helena Margaretha Manurung, Daffha Maullana.
MIT. Third-party components are listed in THIRD_PARTY_NOTICES.md. Game names mentioned are trademarks of their owners; MotionJoy is not affiliated with them.

