WayangDex is a Pokédex-inspired digital encyclopedia web application developed as an undergraduate thesis project. It leverages computer vision to help users identify, collect, and learn about traditional Indonesian shadow puppet (Wayang Kulit) characters in an interactive, gamified environment.
Powered by a custom YOLO model (exported to ONNX format) served via a Python FastAPI backend, WayangDex processes captured photos on-demand to identify characters, unlock detailed profiles, character lore, and progression badges.
| Screen | Description | Preview |
|---|---|---|
| Scan Standby | Home screen displaying scanner status and camera activation option. |  |
| Capture Interface | Camera framing view with shutter trigger and manual camera controls (pinch to zoom, tap to focus). |  |
| Detection Result | Instant bounding box overlay identifying the captured character (e.g., Abimanyu), key traits, and rewarded XP. |  |
| Character Detail | Deep-dive lore sheet containing aliases, character transformations, traits, and audio narration. |  |
| Collection Index | Pokédex-style gallery tracking collection progress (e.g., 1/28 collected) with locked slot previews. |  |
| Explorer Profile | Gamification dashboard tracking Explorer Level, total XP, total scans, and unlocked achievement badges. |  |
- Event-Driven AI Detection: Server-side character identification triggered via photo capture using ONNX Runtime Python.
- Digital Encyclopedia & Lore: Comprehensive background info, character traits, alternate names, and audio narration.
- Gamified Collection System: Pokédex-style index featuring completion tracking across distinct character classes.
- Progression Dashboard: User leveling system, XP rewards, total scan counters, and unlockable achievement badges.
- Backend Framework: FastAPI (Python)
- Computer Vision Model: YOLO (Exported to
.onnx) - Inference Engine: ONNX Runtime (
onnxruntime) - Package & Env Manager:
uv - Frontend Architecture: HTML5 / Server-Side Rendered UI
- License: MIT License
This project is licensed under the MIT License - see the LICENSE file for details.