Data / Repositories / Spider
Spider repositories on GitHub: 12 ranked by stars
12 public repositories tagged Spider are on GitHub, with 55,656 stars between them; 2 had a push in the last 30 days. Most are written in Python (4), JavaScript (3), TypeScript (1). The most starred: Evil0ctal/Douyin_TikTok_Download_API (20,637), lorien/awesome-web-scraping (8,153), BruceDone/awesome-crawler (7,327).
SpecStream is the live web data engine for teams and agents. It reads open-source repositories on GitHub and keeps what it reads, and this page groups that reading by topic. Ask for the repositories you track by topic, language, license or activity, and new ones and changes arrive in your sheet, CRM, Slack or agent.
Every repository, most starred first
| Repository | What it is | Language | Stars | Forks | License | Last push | Read |
|---|---|---|---|---|---|---|---|
| Evil0ctal/Douyin_TikTok_Download_API | 抖音、TikTok 数据采集与无水印视频下载 API，自托管，支持 MCP 调用与 Docker 一键部署。| Self-hosted TikTok & Douyin scraper and no-watermark video downloader — async REST API, MCP server, CLI and web console for… | Python | 20,637 | 2,881 | Apache-2.0 | Oct 2 | Oct 14 |
| lorien/awesome-web-scraping | List of libraries, tools and APIs for web scraping and data processing. | JavaScript | 8,153 | 948 | Sep 19 | Sep 21 | |
| BruceDone/awesome-crawler | A collection of awesome web crawler,spider in different languages | 7,327 | 752 | MIT | Jun 16, 2024 | Oct 11 | |
| wechatsync/Wechatsync | 一键同步文章到多个内容平台，支持今日头条、WordPress、知乎、简书、掘金、CSDN、typecho各大平台，一次发布，多平台同步发布。解放个人生产力 | TypeScript | 6,447 | 1,073 | GPL-3.0 | May 27 | Oct 14 |
| geziyor/geziyor | Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering. | Go | 2,778 | 155 | MPL-2.0 | Jul 2 | Sep 21 |
| spider-rs/spider | Foundational low latency web data collecting in Rust | Rust | 2,769 | 312 | MIT | Oct 1 | Oct 11 |
| lorien/grab | Web Scraping Framework | Python | 2,464 | 274 | MIT | Sep 19, 2025 | Sep 21 |
| howie6879/ruia | Async Python 3.6+ web scraping micro-framework based on asyncio | Python | 1,737 | 185 | Apache-2.0 | Sep 17 | Sep 21 |
| elixir-crawly/crawly | Crawly, a high-level web crawling & scraping framework for Elixir. | Elixir | 1,117 | 125 | Apache-2.0 | Jul 16, 2025 | Sep 21 |
| scrapfly/scrapfly-scrapers | Scalable Python web scraping scripts for +40 popular domains | Python | 1,083 | 205 | Sep 18 | Sep 21 | |
| speed/newcrawler | Free Web Scraping Tool with Java | JavaScript | 585 | 112 | Nov 25, 2023 | Sep 21 | |
| zhuyingda/webster | a reliable high-level web crawling & scraping framework for Node.js. | JavaScript | 559 | 52 | GPL-3.0 | Mar 12, 2025 | Sep 21 |
By language
- Python 4
- JavaScript 3
Related topics
- Crawler 11
- Scraping 7
- Scraper 5
- Web scraping 5
- Crawling 5
- Web crawler 3
- Python 3
- Asyncio 2
- Webscraping 2
- Automation 2
- Headless chrome 2
- FastAPI 1
Read from GitHub by SpecStream between September 21, 2026 and October 14, 2026; each row shows the day its repository was read. Stars and pushes change every day, and the GitHub page is the current one. Forks and archived repositories are left out.