Data / Repositories / Scraping
Scraping repositories on GitHub: 38 ranked by stars
38 public repositories tagged Scraping are on GitHub, with 485,081 stars between them; 8 had a push in the last 30 days. Most are written in Python (17), JavaScript (5), TypeScript (4). The most starred: firecrawl/firecrawl (191,485), D4Vinci/Scrapling (86,357), scrapy/scrapy (64,433).
SpecStream is the live web data engine for teams and agents. It reads open-source repositories on GitHub and keeps what it reads, and this page groups that reading by topic. Ask for the repositories you track by topic, language, license or activity, and new ones and changes arrive in your sheet, CRM, Slack or agent.
Every repository, most starred first
| Repository | What it is | Language | Stars | Forks | License | Last push | Read |
|---|---|---|---|---|---|---|---|
| firecrawl/firecrawl | Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. | TypeScript | 191,485 | 10,124 | AGPL-3.0 | Oct 14 | Oct 14 |
| D4Vinci/Scrapling | An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! | Python | 86,357 | 8,855 | BSD-3-Clause | Oct 8 | Oct 8 |
| scrapy/scrapy | Scrapy, a fast high-level web crawling & scraping framework for Python. | Python | 64,433 | 11,969 | BSD-3-Clause | Sep 21 | Sep 21 |
| ScrapeGraphAI/Scrapegraph-ai | Python scraper based on AI | Python | 31,623 | 3,221 | MIT | Oct 6 | Oct 8 |
| apify/crawlee | Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML… | TypeScript | 25,860 | 1,677 | Apache-2.0 | Sep 21 | Sep 21 |
| code4craft/webmagic | A scalable web crawler framework for Java. | Java | 11,676 | 4,100 | Apache-2.0 | Dec 20, 2025 | Sep 21 |
| apify/crawlee-python | Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files… | Python | 9,542 | 809 | Apache-2.0 | Sep 21 | Sep 21 |
| lorien/awesome-web-scraping | List of libraries, tools and APIs for web scraping and data processing. | JavaScript | 8,153 | 948 | Sep 19 | Sep 21 | |
| alirezamika/autoscraper | A Smart, Automatic, Fast and Lightweight Web Scraper for Python | Python | 7,990 | 827 | MIT | Jul 29 | Sep 21 |
| adbar/trafilatura | Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML | Python | 6,845 | 433 | Apache-2.0 | Sep 21 | Sep 21 |
| sparklemotion/mechanize | Mechanize is a ruby library that makes automated web interaction easy. | Ruby | 4,441 | 477 | MIT | Aug 23 | Sep 21 |
| aapatre/Automatic-Udemy-Course-Enroller-GET-PAID-UDEMY-COURSES-for-FREE | Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY… | Python | 3,292 | 560 | GPL-3.0 | Apr 28 | Sep 21 |
| symfony/panther | A browser testing and web crawling library for PHP and Symfony | PHP | 3,071 | 231 | MIT | Jun 4 | Sep 21 |
| geziyor/geziyor | Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering. | Go | 2,778 | 155 | MPL-2.0 | Jul 2 | Sep 21 |
| spider-rs/spider | Foundational low latency web data collecting in Rust | Rust | 2,769 | 312 | MIT | Oct 1 | Oct 11 |
| brightdata/brightdata-mcp | A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access. | JavaScript | 2,675 | 332 | MIT | Oct 14 | Oct 14 |
| lorien/grab | Web Scraping Framework | Python | 2,464 | 274 | MIT | Sep 19, 2025 | Sep 21 |
| emadehsan/thal | Getting started with Puppeteer and Chrome Headless for Web Scraping | JavaScript | 2,358 | 203 | MIT | Oct 28, 2020 | Sep 21 |
| Jieyab89/OSINT-Cheat-sheet | OSINT cheat sheet, list OSINT tools, wiki, dataset, article, book , red team OSINT for hackers and OSINT tips and OSINT branch. This repository will grow every time will research… | HTML | 2,243 | 373 | MIT | Oct 10 | Oct 11 |
| php-embed/Embed | Get info from any web service or page | PHP | 2,140 | 323 | MIT | Jul 8 | Sep 21 |
| tinyfish-io/agentql | AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly… | Python | 1,464 | 171 | MIT | Sep 18 | Sep 21 |
| raznem/parsera | Lightweight library for scraping web-sites with LLMs | Python | 1,358 | 78 | GPL-2.0 | Dec 17, 2025 | Sep 21 |
| plabayo/rama | modular service framework to move and transform network packets | Rust | 1,236 | 107 | Apache-2.0 | Oct 10 | Oct 11 |
| neon-mmd/websurfx | rocket: An open source alternative to searx which provides a modern-looking :sparkles:, lightning-fast :zap:, privacy respecting :disguised_face:, secure :lock: meta search engine | Rust | 1,197 | 129 | AGPL-3.0 | Sep 8 | Sep 21 |
| elixir-crawly/crawly | Crawly, a high-level web crawling & scraping framework for Elixir. | Elixir | 1,117 | 125 | Apache-2.0 | Jul 16, 2025 | Sep 21 |
| scrapfly/scrapfly-scrapers | Scalable Python web scraping scripts for +40 popular domains | Python | 1,083 | 205 | Sep 18 | Sep 21 | |
| daijro/hrequests | Web scraping for humans | Python | 1,022 | 66 | MIT | Dec 1, 2024 | Sep 21 |
| MorvanZhou/easy-scraping-tutorial | Simple but useful Python web scraping tutorial code. | Jupyter Notebook | 822 | 536 | MIT | Apr 7, 2024 | Sep 21 |
| slotix/dataflowkit | Extract structured data from web sites. Web sites scraping. | Go | 713 | 82 | BSD-3-Clause | Mar 7, 2023 | Sep 21 |
| adrianhajdin/pricewise | Dive into web scraping and build a Next.js 13 eCommerce price tracker within a single video that teaches you data scraping, cron jobs, sending emails, deployment, and more. | TypeScript | 635 | 182 | Jul 6, 2024 | Sep 21 | |
| speed/newcrawler | Free Web Scraping Tool with Java | JavaScript | 585 | 112 | Nov 25, 2023 | Sep 21 | |
| lkuffo/web-scraping | Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup | Python | 393 | 213 | GPL-3.0 | Oct 20, 2025 | Sep 21 |
| alephdata/memorious | Lightweight web scraping toolkit for documents and structured data. | Python | 316 | 64 | MIT | May 20 | Sep 21 |
| oxylabs/Python-Web-Scraping-Tutorial | In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move on to relatively more complex. | Python | 305 | 35 | Sep 24, 2025 | Sep 21 | |
| nuhmanpk/WebScrapper | Powerful Telegram bot for web scraping and crawling. Fast, easy, and loved by thousands! | Python | 211 | 116 | MIT | Mar 19 | Sep 21 |
| tinyfish-io/agentql-mcp | Model Context Protocol server that integrates AgentQL's data extraction capabilities. | TypeScript | 181 | 44 | MIT | Oct 8 | Oct 14 |
| umbrellaDocumentation/Web-Data-Scraper | Web Data Scraper - no-code internet scraping. Extract and export to CSV, Excel, JSON, Google Sheets, and Webhook. | JavaScript | 133 | 116 | Mar 17 | Sep 21 | |
| ScrapeGraphAI/scrapegraph-mcp | ScapeGraph MCP Server | Python | 115 | 38 | MIT | Jul 17 | Oct 14 |
By language
- Python 17
- JavaScript 5
- TypeScript 4
- Rust 3
Related topics
- Crawler 20
- Web scraping 19
- Webscraping 12
- Scraper 11
- Crawling 11
- Python 11
- Web crawler 7
- Automation 7
- Playwright 7
- Spider 7
- AI 6
- Data extraction 6
Read from GitHub by SpecStream between September 21, 2026 and October 14, 2026; each row shows the day its repository was read. Stars and pushes change every day, and the GitHub page is the current one. Forks and archived repositories are left out.