A modern desktop tool for extracting and mirroring websites — clearnet and TOR/onion supported.
SiteMirror is a minimalist, dark-themed desktop application for extracting and mirroring entire websites. Whether you're archiving a public site or crawling onion addresses over TOR, SiteMirror handles it with multi-threaded extraction, real-time monitoring, and a clean native UI.
- Sleek dark theme inspired by modern developer tooling
- Real-time extraction monitoring and live thread activity tracking
- Intuitive, responsive layout built with PyQt6
- Single-page or deep recursive crawling
- Full asset downloading — CSS, JS, images, and more
- Link extraction and site mapping
- TOR/Onion site support via
stem
- Multi-threaded extraction with configurable worker count
- Headless browser support for JavaScript-rendered content
- Cookie and session management
robots.txtcompliance toggle- Customizable crawl depth
- Live logs with filtering and search
- Thread monitor showing active workers
- Progress tracking with download statistics
- Configurable output directory
- Python
3.9or higher - Internet connection (+ TOR for onion sites)
# 1. Clone the repository
git clone https://github.com/yourusername/SiteMirror.git
cd SiteMirror
# 2. Create and activate a virtual environment
python -m venv venv
source venv/bin/activate # macOS/Linux
venv\Scripts\activate # Windows
# 3. Install dependencies
pip install -r requirements.txt
# 4. Launch the app
python main.py- Enter URL — Paste a website or
.onionaddress into the extraction panel - Configure Options — Set crawl depth, choose assets to download, and toggle advanced settings
- Start Extraction — Hit Start Extraction to begin
- Monitor Progress — Watch real-time logs and active thread count
- Access Files — All downloaded content is saved to your configured output directory
SiteMirror/
├── backend/
│ ├── core/ # Crawler engine & proxy management
│ ├── threading/ # Worker pool for multi-threading
│ ├── storage/ # File download & storage handling
│ └── logs/ # Logging system
├── ui/
│ ├── main_window.py # Main application window
│ ├── panels/ # UI panels and components
│ ├── widgets/ # Custom PyQt widgets
│ └── styles/ # Theme and styling
├── config/ # Configuration files
├── data/ # Extracted data output
└── main.py # Entry point
| Layer | Technology |
|---|---|
| UI | PyQt6 / PySide6 |
| HTTP | aiohttp (async) |
| Parsing | BeautifulSoup4 |
| Browser Automation | Playwright (optional) |
| TOR Support | stem |
| Async Runtime | asyncio |
| Element | Color |
|---|---|
| Background | #0B0F17 |
| Card | #151B2C |
| Primary Accent | #7C4DFF |
| Success | #22C55E |
| Warning | #F59E0B |
| Error | #EF4444 |
- Async I/O with
asyncioandaiohttpfor minimal blocking - Configurable thread pool for large-scale extractions
- Memory-efficient virtualized log viewer
- Optimized for both quick single-page grabs and deep recursive crawls
Contributions are welcome! Feel free to open an issue or submit a pull request.
This project is licensed under the MIT License — see the LICENSE file for details.
SiteMirror is intended for educational and authorized use only. Always respect a website's Terms of Service and robots.txt rules. The author is not responsible for any misuse of this tool.
Made with 🖤 by Farisxdev
Found this useful? Drop a ⭐ on GitHub — it helps a lot!