🎮 Adversarial Game AI Framework

Train AI opponents that learn YOUR playstyle and exploit your weaknesses

An adversarial game AI framework that creates "impossibly hard" opponents through opponent modeling - not superhuman reflexes, but smarter strategy. The AI predicts your next move and counters it, getting progressively harder as you play.

---

🎯 Core Concept

┌─────────────────────────────────────────────────────────────┐
│  1. YOU PLAY → System records your patterns                 │
│  2. LSTM MODEL → Learns to predict your next action         │
│  3. ADVERSARIAL AI → Uses predictions to counter your moves │
│  4. ITERATE → AI keeps learning your adaptations            │
└─────────────────────────────────────────────────────────────┘

Each session makes the AI smarter about YOUR specific strategies.

---

🚀 Quick Start

Prerequisites

• Python 3.8+
• PyTorch 2.0+

Installation

# Clone the repository
git clone https://github.com/Kukyos/GameAI.git
cd GameAI

# Install dependencies
pip install -r requirements.txt

Run the Fighting Game

# Test the environment
python src/environments/simple_fighter.py

# Collect gameplay data
python src/player_model/fighting_game_collector.py

# Run the demo (requires trained models)
python demo/adversarial_demo.py

Controls

Key	Action
Q	Light Attack
W	Block
E	Dodge
A	Move Back
D	Move Forward

---

📁 Project Structure

GameAI/
├── src/
│   ├── environments/          # Custom fighting game environments
│   │   ├── simple_fighter.py  # Main 2D fighter with frame data
│   │   └── fighting_game.py   # Gymnasium-compatible wrapper
│   │
│   ├── player_model/          # Player behavior prediction
│   │   ├── model.py           # LSTM prediction model
│   │   ├── trainer.py         # Training pipeline
│   │   └── fighting_game_collector.py  # Data collection
│   │
│   ├── adversarial_policy/    # AI that exploits predictions
│   │   ├── policy.py          # Actor-Critic with opponent modeling
│   │   └── trainer.py         # PPO training with predictions
│   │
│   ├── bc/                    # Behavior Cloning
│   │   └── train_bc.py        # Clone human demonstrations
│   │
│   ├── rl/                    # Reinforcement Learning
│   │   ├── ppo_custom.py      # Custom PPO implementation
│   │   └── train_ppo_sb3.py   # Stable-Baselines3 integration
│   │
│   ├── models/                # Neural network architectures
│   │   ├── bc_model.py        # BC policy network
│   │   ├── policy.py          # Actor-Critic networks
│   │   └── style_encoder.py   # Player style embedding
│   │
│   ├── data/                  # Data handling
│   │   ├── collector.py       # Generic data collection
│   │   ├── dataset.py         # PyTorch Dataset classes
│   │   └── preprocess.py      # Data preprocessing
│   │
│   ├── demo/                  # Visualization & demos
│   │   ├── pygame_renderer.py # Pygame visualization
│   │   └── streamlit_demo.py  # Web-based demo
│   │
│   └── eval/                  # Evaluation tools
│       └── evaluate.py        # Model evaluation metrics
│
├── demo/                      # Playable demo scripts
│   └── adversarial_demo.py    # Play against the AI
│
├── checkpoints/               # Pre-trained models
│   ├── player_model.pth       # Trained LSTM predictor
│   └── adversarial_policy.pth # Trained adversarial agent
│
├── data/raw/                  # Collected gameplay sessions
├── envs/                      # Gymnasium wrappers
└── scripts/                   # Training & eval scripts

---

🧠 Technical Architecture

Player Prediction Model (LSTM)

Input:  Last N frames of player actions + game state
Output: Probability distribution over next action

• Bidirectional LSTM with attention
• Trained on collected gameplay data
• Achieves ~70% prediction accuracy on habitual patterns

Adversarial Policy (PPO + Opponent Modeling)

Input:  Game state + Player prediction probabilities
Output: Optimal counter-action

• Actor-Critic architecture
• Reward shaped by exploiting predicted actions
• Human-like constraints (reaction delay, stamina, cooldowns)

Fighting Game Environment

• Frame-based combat with startup, active, recovery frames
• Hitstun/Blockstun mechanics
• Stamina system prevents spam
• Reaction delay (8-12 frames) for human-like AI

---

📊 Current Status

• Custom 2-player fighter environment with proper frame data
• Data collection system with keyboard controls
• LSTM player prediction model
• Adversarial PPO policy architecture
• PPO trainer with opponent modeling
• Demo interface with Pygame rendering
• Full adversarial training loop integration
• Iterative improvement pipeline
• Unity 2D fighter integration (Phase 2)

---

🔧 Configuration

Key training parameters in the respective trainer files:

# Player Model (src/player_model/trainer.py)
sequence_length = 30
hidden_size = 128
learning_rate = 1e-3

# Adversarial Policy (src/adversarial_policy/trainer.py)
prediction_weight = 0.3  # How much to weight predictions
ppo_epochs = 10
clip_epsilon = 0.2

---

🎓 How It Works

Phase 1: Data Collection

Player gameplay is recorded as sequences:

(state_t, action_t, state_t+1, action_t+1, ...)

Phase 2: Player Modeling

LSTM learns temporal patterns:

P(action_t | state_t, action_{t-1}, ..., action_{t-N})

Phase 3: Adversarial Training

PPO agent receives augmented observations:

obs = [game_state, predicted_player_action_probs]
reward = game_reward + exploitation_bonus

Phase 4: Iteration

After each play session:

1. Collect new data against current AI
2. Retrain player model on updated data
3. Fine-tune adversarial policy
4. Repeat

---

🤝 Contributing

Contributions are welcome! Areas that need work:

1. Full training pipeline - Connect all components end-to-end
2. Better reward shaping - Tune exploitation vs. winning balance
3. Unity integration - Port to a proper game engine
4. More game environments - Extend beyond fighting games

---

📜 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

🔗 References

• PPO Paper - Proximal Policy Optimization
• Opponent Modeling - Learning with Opponent-Learning Awareness
• Fighting Game AI - Deep RL for Fighting Games

---

Built with PyTorch, Gymnasium, and PPO