Data / Repositories / Web scraper
Web scraper repositories on GitHub: 9 ranked by stars
9 public repositories tagged Web scraper are on GitHub, with 416,248 stars between them; 4 had a push in the last 30 days. Most are written in Python (5), TypeScript (1), Rust (1). The most starred: firecrawl/firecrawl (191,485), Panniantong/Agent-Reach (97,255), D4Vinci/Scrapling (86,357).
SpecStream is the live web data engine for teams and agents. It reads open-source repositories on GitHub and keeps what it reads, and this page groups that reading by topic. Ask for the repositories you track by topic, language, license or activity, and new ones and changes arrive in your sheet, CRM, Slack or agent.
Every repository, most starred first
| Repository | What it is | Language | Stars | Forks | License | Last push | Read |
|---|---|---|---|---|---|---|---|
| firecrawl/firecrawl | Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. | TypeScript | 191,485 | 10,124 | AGPL-3.0 | Oct 14 | Oct 14 |
| Panniantong/Agent-Reach | Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees. | Python | 97,255 | 8,509 | MIT | Oct 8 | Oct 14 |
| D4Vinci/Scrapling | An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! | Python | 86,357 | 8,855 | BSD-3-Clause | Oct 8 | Oct 8 |
| ScrapeGraphAI/Scrapegraph-ai | Python scraper based on AI | Python | 31,623 | 3,221 | MIT | Oct 6 | Oct 8 |
| BruceDone/awesome-crawler | A collection of awesome web crawler,spider in different languages | 7,327 | 752 | MIT | Jun 16, 2024 | Oct 11 | |
| us/crw | Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl… | Rust | 1,059 | 84 | AGPL-3.0 | Sep 21 | Sep 21 |
| je-suis-tm/web-scraping | Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and… | Python | 911 | 190 | Apache-2.0 | Feb 1, 2022 | Sep 21 |
| umbrellaDocumentation/Web-Data-Scraper | Web Data Scraper - no-code internet scraping. Extract and export to CSV, Excel, JSON, Google Sheets, and Webhook. | JavaScript | 133 | 116 | Mar 17 | Sep 21 | |
| Sharan-Kumar-R/Custom-MCP-Server | MCP server for scraping LinkedIn, Facebook, Instagram profiles and Google search. | Python | 98 | 76 | Jun 27, 2025 | Oct 14 |
By language
- Python 5
Related topics
- Web scraping 7
- Crawler 6
- Data extraction 5
- Web crawler 5
- AI agents 4
- Scraping 4
- Webscraping 4
- AI 3
- LLM 3
- Markdown 3
- Scraper 3
- MCP 3
Read from GitHub by SpecStream between September 21, 2026 and October 14, 2026; each row shows the day its repository was read. Stars and pushes change every day, and the GitHub page is the current one. Forks and archived repositories are left out.