A simple trap for web crawlers
-
Updated
Aug 2, 2023 - Rust
A simple trap for web crawlers
Autonomous web form inspector and schema auto-mapper matching input attributes, autocomplete hints, and regex validations to agent state.
Autonomous web form inspector and schema auto-mapper matching input attributes, autocomplete hints, and regex validations to agent state.
Web URL canonicalization and anti-crawler trap detector identifying circular paths, infinite paginations, and tracking parameter bloat.
Zero-dependency HTML DOM semantic tree pruner compressing web markup into accessibility-friendly clean trees with 80%+ token reduction.
Execution trajectory evaluator computing action-level precision, recall, and Levenshtein sequence edit distance against gold benchmarks.
Zero-dependency HTML DOM semantic tree pruner compressing web markup into accessibility-friendly clean trees with 80%+ token reduction.
Zero-dependency Markdown table to structured JSON transformer with automated column type inference and format normalization.
Web URL canonicalization and anti-crawler trap detector identifying circular paths, infinite paginations, and tracking parameter bloat.
Execution trajectory evaluator computing action-level precision, recall, and Levenshtein sequence edit distance against gold benchmarks.
Zero-dependency Markdown table to structured JSON transformer with automated column type inference and format normalization.
Modern, customizable web honeypot server detecting malicious web crawlers, AI scrapers, and attackers via fake credentials, canary tokens, spider traps, and a real-time dashboard.
To associate your repository with the crawler-trap topic, visit your repo's landing page and select "manage topics."