Best web scraping tools 2026

1. Create a key under Agents & API in the workspace. A new account gets $10 of credit, no card, which covers 12,500 pages.

2. Put the URLs you scrape today in the call. Each result shows its status (ok, blocked, not_found, empty), where it was read from (live, site or archive) and the length of its text. Only ok pages are charged:

curl -s https://api.specstream.ai/v1/extract \
  -H "Authorization: Bearer $SPECSTREAM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://en.wikipedia.org/wiki/Web_scraping", "https://www.python.org/", "https://www.w3.org/TR/PNG/"], "format": "markdown"}' \
  | jq '(.data.results[] | {url, status, read_from, chars: (.text // "" | length)}), .data.price_usd'

3. On a scraping API now? Change the address and keep the call. For ScraperAPI, in Python:

import os
import requests

URL = "https://api.specstream.ai/scraperapi"  # was https://api.scraperapi.com

reply = requests.get(
    URL,
    params={"api_key": os.environ["SPECSTREAM_API_KEY"], "url": "https://en.wikipedia.org/wiki/Web_scraping", "output_format": "markdown"},
    timeout=70,
)
reply.raise_for_status()
print(reply.text[:500])

4. The ScrapingBee, ZenRows, Firecrawl and Crawl4AI guides list every parameter. For records rather than pages, describe them in Deep Search.