From 8674da61acedf4af6afe228ca30142ea12ddf573 Mon Sep 17 00:00:00 2001 From: Pascal Essiembre Date: Tue, 1 Sep 2026 21:39:34 -0400 Subject: [PATCH] Add Norconex Crawler to Web Crawling Signed-off-by: Pascal Essiembre --- README_SOURCE.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README_SOURCE.md b/README_SOURCE.md index 12ab52b4..177e1d11 100644 --- a/README_SOURCE.md +++ b/README_SOURCE.md @@ -1242,6 +1242,7 @@ _Libraries that analyze the content of websites._ - [Apache Nutch](https://github.com/apache/nutch) - Highly extensible, highly scalable web crawler for production environments. - [crawlberg](https://github.com/xberg-io/crawlberg) - Crawls and scrapes websites through a Java binding with Markdown conversion and optional browser rendering. - [jsoup](https://github.com/jhy/jsoup) - Scrapes, parses, manipulates and cleans HTML. +- [Norconex Crawler](https://github.com/Norconex/crawler) - Modular web and file-system crawler with official Docker images and a browser-based configurator. - [StormCrawler](https://github.com/apache/stormcrawler) - SDK for building low-latency and scalable web crawlers. - [webmagic](https://github.com/code4craft/webmagic) - Scalable crawler with downloading, url management, content extraction and persistent.