# Spider's documented calls against SpecStream

Date: 2026-11-10. Client: Spider's documented curl calls (API reference read 2026-11-10).

## The one line a user changes

Before:

```
https://api.spider.cloud
```

After:

```
https://api.specstream.ai/spider
```

spider-client 0.1.93 writes Spider's own address into every call, so the calls here are Spider's documented requests with the address and the key changed. A scrape is one Extract API call; a page with no text is not charged. A search is one web search. Spider's crawl (`/crawl`) is not served at this address.

## Request parameters

Honoured means the check in the Measured column passed on the reply. Accepted, not applied means status 200 and the reply has the form of the same call without the parameter. Refused means the vendor's error body and status.

Counts: 17 honoured, 20 accepted, not applied, 3 refused.

| Parameter | Sent | Verdict | Measured |
|---|---|---|---|
| `scrape url` | `https://en.wikipedia.org/wiki/Web_scraping` | honoured | status 200; a list of one page, content the page's HTML (238401 characters) |
| `scrape (the /v1/ path)` | `/v1/scrape` | honoured | status 200; the same list |
| `scrape url (two, by commas)` | two URLs | honoured | status 200; 2 pages |
| `scrape return_format` | `markdown` | honoured | status 200; content is markdown |
| `scrape return_format` | `text` | honoured | status 200; content has no markdown marks and no tags |
| `scrape return_format` | `empty` | honoured | status 200; content null |
| `scrape return_format (a list)` | `[markdown, raw]` | honoured | status 200; content in the first format |
| `scrape return_page_links` | `true` | honoured | status 200; 335 links, each a string |
| `scrape metadata` | `true` | honoured | status 200; metadata.title 'Web scraping - Wikipedia' |
| `scrape url (a page the site answers 404)` | a missing page | honoured | status 200; status 404, costs None |
| `scrape request` | `"smart"` | accepted, not applied | status 200; the baseline's form |
| `scrape readability` | `true` | accepted, not applied | status 200; the baseline's form |
| `scrape anti_bot` | `true` | accepted, not applied | status 200; the baseline's form |
| `scrape stealth` | `true` | accepted, not applied | status 200; the baseline's form |
| `scrape proxy_enabled` | `true` | accepted, not applied | status 200; the baseline's form |
| `scrape cache` | `false` | accepted, not applied | status 200; the baseline's form |
| `scrape wait_for` | `{"idle_network": {"timeout": {"secs": 1, "nanos": 0}}}` | accepted, not applied | status 200; the baseline's form |
| `scrape css_extraction_map` | `{}` | accepted, not applied | status 200; the baseline's form |
| `scrape filter_output_main_only` | `true` | accepted, not applied | status 200; the baseline's form |
| `scrape respect_robots` | `false` | accepted, not applied | status 200; the baseline's form |
| `scrape full_resources` | `true` | accepted, not applied | status 200; the baseline's form |
| `scrape block_ads` | `true` | accepted, not applied | status 200; the baseline's form |
| `scrape country_code` | `"de"` | accepted, not applied | status 200; the baseline's form |
| `scrape locale` | `"de-DE"` | accepted, not applied | status 200; the baseline's form |
| `scrape (no url)` | `{}` | refused | status 400; an error body |
| `search search` | `web scraping history` | honoured | status 200; 10 results with no count named |
| `search (the /v1/ path)` | `/v1/search` | honoured | status 200; 3 results |
| `search search_limit` | `4` | honoured | status 200; 4 results |
| `search num` | `5` | honoured | status 200; 5 results |
| `search page` | `2` | honoured | status 200; results apart from page 1's |
| `search query operator` | `site:wikipedia.org` | honoured | status 200; every result on the site |
| `search country, language` | `de`, `de` | accepted, not applied | status 200; 0 results on .de hosts, 0 without them |
| `search tbs` | `qdr:w` | honoured | status 200; the search ran 'news after:2026-11-03' |
| `search fetch_page_content` | `false` | accepted, not applied | status 200; the baseline's form |
| `search location` | `"Berlin"` | accepted, not applied | status 200; the baseline's form |
| `search website_limit` | `2` | accepted, not applied | status 200; the baseline's form |
| `search quick_search` | `true` | accepted, not applied | status 200; the baseline's form |
| `search auto_pagination` | `false` | accepted, not applied | status 200; the baseline's form |
| `search (no search)` | `{}` | refused | status 400; an error body |
| `api key (wrong key)` | a key that is not one | refused | status 401; error body ['code', 'error'] |

## Response fields

Present means the field holds a value in the replies to the calls above. Empty means it is there with no value. Absent means it is not in the reply.

Counts: 10 present, 0 empty, 1 absent.

| Field | State | Measured |
|---|---|---|
| `scrape [].url` | present | every page |
| `scrape [].content` | present | every page |
| `scrape [].status` | present | every page |
| `scrape [].error` | present | every page |
| `scrape [].costs` | present | every page |
| `scrape [].links` | present | every page |
| `scrape [].metadata` | present | every page |
| `scrape [].duration_elapsed_ms` | absent | every page |
| `search content[].title` | present | every result |
| `search content[].description` | present | every result |
| `search content[].url` | present | every result |