Skip to content

About

Play PC games with your body: a webcam motion controller for Windows. Local pose tracking (MediaPipe), game presets, floating panel. Prototype.

Topics

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

MotionJoy logo

MotionJoy

Play PC games with your body. Your webcam is the controller.

CI Release License: MIT Platform: Windows Status: prototype

Download · Docs · Report a bug · Bahasa Indonesia

MotionJoy turns body movement into keyboard input. Lean left or right, jump, or crouch in front of your webcam and the game you are playing receives the matching arrow key. It needs no extra hardware, runs fully on your computer, and uses ready-made presets for games.

Status: prototype. MotionJoy is an early student project (JDH Group, Institut Teknologi Indonesia). It works end to end, but gesture accuracy has not been formally measured and some gestures (especially crouch) still need tuning.

Floating panel over a game Setup screen

Features

  • Webcam only. No sensors, no controller. Pose tracking runs locally with MediaPipe.
  • Game presets. Subway Surfers and Chrome Dino included. Add your own with a small JSON file (docs).
  • Floating panel that does not steal focus. Always on top of the game; resize it, hide the camera, or move it anywhere.
  • Sensitivity you can tune live. Low / Normal / High / Custom, with sliders for Lean, Jump and Crouch while you play.
  • Automatic calibration. Stand in your play position and the app calibrates after a short countdown. One click for a new player.
  • Lighting check. Warns when the room is too dark, too bright, or you are backlit.
  • Private by design. No video is recorded or sent anywhere. The app works offline (details).

Quick start

Download (recommended)

  1. Open the Releases page and download one of:
    • MotionJoy-Setup-<version>.exe: installer
    • MotionJoy-Portable-<version>.exe: runs without installing
  2. Windows may show "Windows protected your PC" because the app is not code-signed yet. Click More info, then Run anyway. You can compare the file with SHA256SUMS.txt from the same release.
  3. Start the game you want to play first, then start MotionJoy.

Requirements: Windows 10 or 11 (64-bit) and a webcam. Run MotionJoy and the game at the same privilege level (both normal, not Administrator).

How to play

  1. Setup: stand 1.5 to 2 m from the camera so your head to hips is visible. Check the lighting badge.
  2. Choose game: pick a game. Press Test keys (5 s), then click the game window to confirm it receives the keys.
  3. Start playing: the panel shrinks and floats above the game. Click the game window. After the countdown, MotionJoy calibrates (stand still for 3 seconds).
  4. Play: lean, jump, crouch. Use the icons on the panel:
Icon What it does
Rotate Recalibrate / new player
Pause Pause or resume key sending
Sliders Sensitivity (Low, Normal, High, Custom)
Eye Hide or show the camera preview
Gear Settings, session log, about
X Quit

Drag the panel header to move it and the bottom-right corner to resize it.

Gestures

Gesture Default key (Subway Surfers)
Lean left / right Left / Right arrow
Jump Up arrow
Crouch Down arrow

Run from source

git clone https://github.com/MuhammadZaidan1/motionjoy-controller-desktop.git
cd motionjoy-controller-desktop
npm install
npm start

Key sending only works on Windows. The engine and tests run on any OS: npm test.

Build the Windows app

On a Windows machine:

npm install
npm run dist

Output goes to dist/: an installer (MotionJoy-Setup-*.exe) and a portable exe (MotionJoy-Portable-*.exe). If the build fails with a symbolic-link error, see Troubleshooting.

Releases (maintainers)

Releases are built by GitHub Actions. Update version in package.json, commit, then:

git tag v0.9.0-prototype
git push origin v0.9.0-prototype

The workflow builds on Windows, runs the tests, and publishes a Release with the executables and checksums. The tag must match package.json (v + version).

Project structure

src/
  main.js, preload.js, keys.js   Electron main process, bridge, key injection (Windows SendInput)
  engine/                        Pure logic: metrics, gestures, presets, sensitivity, lighting
  ui/                            Interface (HTML, CSS, ES modules) and design system
  vendor/                        Bundled MediaPipe runtime and pose models (offline)
presets/                         Game presets (JSON)
test/                            Unit tests for the engine
docs/                            Architecture, presets, privacy, troubleshooting, booth guide
build/                           App icon
.github/                         CI, release workflow, issue and PR templates

More detail in docs/ARCHITECTURE.md.

Limitations

  • Prototype: not formally evaluated; do not treat it as an accuracy claim.
  • Crouch detection depends on camera angle and framing and still needs tuning.
  • Windows only. Games with anti-cheat or elevated rights may ignore injected keys.
  • Keys are sent to whichever window has focus. Click the game window before playing.
  • The executable is not code-signed, so SmartScreen shows a warning.

Roadmap

  • Measure gesture reliability with real players and tune defaults.
  • More game presets and an in-app preset editor.
  • Code signing and auto-update.
  • Optional gamepad emulation.

Contributing

Issues and pull requests are welcome. See CONTRIBUTING.md.

Team

JDH Group, Institut Teknologi Indonesia: Muhammad Zaidan Fadlurrahman, Helena Margaretha Manurung, Daffha Maullana.

License

MIT. Third-party components are listed in THIRD_PARTY_NOTICES.md. Game names mentioned are trademarks of their owners; MotionJoy is not affiliated with them.

About

Play PC games with your body: a webcam motion controller for Windows. Local pose tracking (MediaPipe), game presets, floating panel. Prototype.

Topics

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages