Spider (spider.cloud) compatible

The call, before and after

# Before
curl -X POST https://api.spider.cloud/scrape \
  -H "Authorization: Bearer $SPIDER_API_KEY" -H "Content-Type: application/json" \
  -d '{"url": "https://en.wikipedia.org/wiki/Web_scraping", "return_format": "markdown"}'

# After
curl -X POST https://api.specstream.ai/spider/scrape \
  -H "Authorization: Bearer $SPECSTREAM_API_KEY" -H "Content-Type: application/json" \
  -d '{"url": "https://en.wikipedia.org/wiki/Web_scraping", "return_format": "markdown"}'

The reply is Spider's: a list with one object a page, [{"url": "...", "content": "# Web scraping ...", "status": 200, "error": null, "costs": {"total_cost": 0.0008}}].

spider-client 0.1.93 writes https://api.spider.cloud into every call and takes no other address, so the switch is in your own HTTP call: the address and the key. The bodies Spider documents run unchanged, and the /v1/ paths answer too.

What the route reads, and what it refuses

Scrape, applied: url, one or several by commas, return_format as markdown, text, raw, empty or a list (the first format wins), return_page_links and metadata. A page the site answers 404 comes back with status 404 and costs null.

Search, applied: search, search_limit or num, page, tbs such as qdr:w, and operators such as site:. A search that names no count gets 10 results.

Refused, in Spider's own {"error", "code"} body: a scrape with no url or a search with no search gets 400, and a wrong key gets 401. Every parameter was measured on production on Nov 10, 2026, and the full pass table lists each one and its result.

Spider's POST /crawl answers every page in one reply, and it is not served at /spider/crawl. A crawl on SpecStream is a job you start and then read: Firecrawl's /v2/crawl on the same key, at $0.0008 a page.

Every field that comes back

A scrape returns, for each page, url, content in the format asked, status with the page's own HTTP status, error, costs, links with return_page_links and metadata with metadata: true. A search returns content[] with title, description and url. The route does not return duration_elapsed_ms.

One price a page, and a failed page is free

On Spider a request is billed by the gigabytes of content it pulls and the CPU minutes it uses. On SpecStream every page that gives text costs $0.0008, which is $0.80 per 1,000, a search of up to 10 results costs $0.005, and a busy call gets 429 with Retry-After and charges nothing. Credit you buy never expires.

Scored on the official benchmarks

The SpecStream Extract API scores 0.908 F1 on WCXB. The Search API scores 97.9 F-score on all 1,000 SimpleQA Verified questions, 978 of them correct, and 64.0 item F1 and 43.4 row F1 on all 200 WideSearch tasks. Every score and its method is on the benchmarks page.

You decide what is kept, on each key

Create the key with zero_retention and a call made with it keeps no request text and no answer after the reply. spend_cap_cents is the most the key can spend.

curl -X POST https://api.specstream.ai/v1/account/keys \
  -H "Authorization: Bearer $SPECSTREAM_API_KEY" -H "Content-Type: application/json" \
  -d '{"name": "production", "scopes": ["account:read", "finds:spend"], "spend_cap_cents": 500000, "zero_retention": true}'

Pages, search and rows on one key

The same key reads up to 20 pages or documents a call with the Extract API, runs crawls and site maps at https://api.specstream.ai/firecrawl, and searches the web at $5 per 1,000 searches. Deep Search returns every match for a request in plain words as structured rows, at 5¢, 15¢ or $1 a row, with a source URL on every value.

Firecrawl

Scrape, search, map and crawl in credits: Hobby is $19 a month for 5,000 credits. SpecStream answers Firecrawl's v2 calls at https://api.specstream.ai/firecrawl, and $19 reads 23,750 pages.

Crawl4AI

An open-source crawler you host and run yourself. SpecStream answers the Crawl4AI server's /crawl and /md at https://api.specstream.ai/crawl4ai, with no server to keep up.

ScrapingBee

A page API on monthly plans: Hobby is $19 a month for 75,000 credits, and a call costs 1 to 75 credits. SpecStream answers ScrapingBee's call at https://api.specstream.ai/scrapingbee/api/v1/ at $0.80 per 1,000 pages.

Tavily

Search, extract and map billed in credits at $0.008 a credit on pay as you go. SpecStream answers Tavily's calls at https://api.specstream.ai/tavily at $0.005 a search.