# Firecrawl map and crawl: the Python client against SpecStream

Date: 2026-11-10. Client: firecrawl-py 4.55.0.

## The one line a user changes

Before:

```
Firecrawl(api_key="fc-...")
```

After:

```
Firecrawl(api_key=KEY, api_url="https://api.specstream.ai/firecrawl")
```

The same address serves Firecrawl's search and scrape. A map names the start address first, as Firecrawl's does, and is one page read when the site gives a link of its own; a crawl charges one page read for each page that gives text. The SDK's waiter (`crawl`) polls the job to its end and follows `next`.
The JavaScript client @mendable/firecrawl-js 4.52.0, with `new Firecrawl({ apiKey: KEY, apiUrl: "https://api.specstream.ai/firecrawl" })`, passes 6 of 6 checks: map, crawl with its waiter to the end, startCrawl, getCrawlStatus, cancelCrawl, and the 404 of an unknown id (`js_results.json`).

## Request parameters

Honoured means the check in the Measured column passed on the reply. Accepted, not applied means status 200 and the reply has the form of the same call without the parameter. Refused means the vendor's error body and status.

Counts: 20 honoured, 13 accepted, not applied, 2 refused.

| Parameter | Sent | Verdict | Measured |
|---|---|---|---|
| `map url` | `https://quotes.toscrape.com/` | honoured | status 200; 47 links, all on the site |
| `map url (a page with no links)` | `https://example.com` | honoured | status 200; 1 link, the start address |
| `map limit` | `3` | honoured | status 200; 3 links |
| `map search` | `einstein` | honoured | status 200; first link https://quotes.toscrape.com/author/Albert-Einstein |
| `map ignoreQueryParameters (default true)` | not sent | honoured | status 200; 20 links, none with a query |
| `map ignoreQueryParameters` | `false` | honoured | status 200; 149 links with a query |
| `map sitemap` | `only` | honoured | status 200; 1020 links from the sitemaps |
| `map sitemap` | `skip` | honoured | status 200; 37 links from the page, 1036 with the sitemaps |
| `map includeSubdomains` | `false` | honoured | status 200; every link on the start host |
| `map timeout` | `30000` | accepted, not applied | status 200; 47 links, the baseline's form |
| `map location` | `DE`, `de-DE` | accepted, not applied | status 200; 47 links, the baseline's form |
| `crawl url` | `https://quotes.toscrape.com/` | honoured | status 200; status completed, 3 pages, the start page first |
| `crawl limit` | `3` | honoured | status 200; 3 pages |
| `crawl includePaths` | `^/author/.*` | honoured | status 200; 2 pages after the start page, all under /author/ |
| `crawl excludePaths` | `^/tag/.*`, `^/login` | honoured | status 200; 4 pages, none excluded |
| `crawl maxDiscoveryDepth` | `0` | honoured | status 200; 1 pages |
| `crawl crawlEntireDomain` | `true` from /tag/love/ | honoured | status 200; 2 pages outside the folder; without it 0 of 3 |
| `crawl scrapeOptions.formats` | `markdown`, `links`, `rawHtml` | honoured | status 200; the first page has all three |
| `crawl scrapeOptions.formats` | `html` | honoured | status 200; the first page has html |
| `crawl sitemap` | `only` | accepted, not applied | status 200; status completed, 2 pages |
| `crawl allowSubdomains` | `true` | accepted, not applied | status 200; status completed, 2 pages |
| `crawl allowExternalLinks` | `true` | accepted, not applied | status 200; status completed, 2 pages |
| `crawl ignoreRobotsTxt` | `true` | accepted, not applied | status 200; status completed, 2 pages |
| `crawl ignoreQueryParameters` | `true` | accepted, not applied | status 200; status completed, 2 pages |
| `crawl delay` | `1` | accepted, not applied | status 200; status completed, 2 pages |
| `crawl maxConcurrency` | `2` | accepted, not applied | status 200; status completed, 2 pages |
| `crawl prompt` | `only author pages` | accepted, not applied | status 200; status completed, 2 pages |
| `crawl webhook` | a URL | accepted, not applied | status 200; status completed, 2 pages |
| `crawl zeroDataRetention` | `true` | accepted, not applied | status 200; status completed, 2 pages |
| `crawl scrapeOptions.onlyMainContent` | `false` | accepted, not applied | status 200; status completed, 2 pages |
| `start_crawl` | limit 12 | honoured | status 200; an id and its status URL |
| `cancel_crawl (DELETE)` | a running job | honoured | status 200; returned True |
| `get_crawl_status of a cancelled job` | its id | honoured | status 200; status cancelled |
| `get_crawl_status` | an unknown id | refused | status 404; FirecrawlError |
| `api key` | a key that is not one | refused | status 401; UnauthorizedError |

## Response fields

Present means the field holds a value in the replies to the calls above. Empty means it is there with no value. Absent means it is not in the reply.

Counts: 24 present, 0 empty, 8 absent.

| Field | State | Measured |
|---|---|---|
| `map success` | present | every map |
| `map links` | present | every map |
| `map links[].url` | present | every link |
| `map links[].title` | absent | every link |
| `map links[].description` | absent | every link |
| `crawl success` | present | every crawl start |
| `crawl id` | present | every crawl start |
| `crawl url` | present | every crawl start |
| `status success` | present | every status read |
| `status status` | present | every status read |
| `status total` | present | every status read |
| `status completed` | present | every status read |
| `status creditsUsed` | present | every status read |
| `status expiresAt` | present | every status read |
| `status next` | present | every status read |
| `status data` | present | every status read |
| `status createdAt` | absent | every status read |
| `status completedAt` | absent | every status read |
| `status duration` | absent | every status read |
| `status data[].markdown` | present | every page |
| `status data[].html` | present | every page |
| `status data[].rawHtml` | present | every page |
| `status data[].links` | present | every page |
| `status data[].screenshot` | absent | every page |
| `status data[].metadata` | present | every page |
| `status data[].metadata.title` | present | every page |
| `status data[].metadata.description` | absent | every page |
| `status data[].metadata.language` | absent | every page |
| `status data[].metadata.sourceURL` | present | every page |
| `status data[].metadata.url` | present | every page |
| `status data[].metadata.statusCode` | present | every page |
| `status data[].metadata.error` | present | every page |