dzhng/deep-research without a Firecrawl key: set FIRECRAWL_BASE_URL

dzhng/deep-research does all of its web work through one call: a Firecrawl search that also returns each result's page as Markdown. The address of that call is a setting, FIRECRAWL_BASE_URL. SpecStream answers Firecrawl's v1 search at https://api.specstream.ai/firecrawl, so the switch is two lines in .env.local:

FIRECRAWL_KEY="YOUR_SPECSTREAM_API_KEY"
FIRECRAWL_BASE_URL="https://api.specstream.ai/firecrawl"

No file in src changes, and there is no Firecrawl server to run. The rest of this post shows where the call sits, what comes back, and what a run costs.

Where Firecrawl sits in the code

src/deep-research.ts builds one Firecrawl client from two environment variables:

const firecrawl = new FirecrawlApp({
  apiKey: process.env.FIRECRAWL_KEY ?? '',
  apiUrl: process.env.FIRECRAWL_BASE_URL,
});

Each search query the model writes becomes one call:

const result = await firecrawl.search(serpQuery.query, {
  timeout: 15000,
  limit: 5,
  scrapeOptions: { formats: ['markdown'] },
});

The lock file pins @mendable/firecrawl-js 1.16.0. That version of the SDK posts to the address it is given followed by /v1/search, and sends the key as Authorization: Bearer. With the two lines above, every search goes to https://api.specstream.ai/firecrawl/v1/search with your SpecStream key. The code never calls Firecrawl's scrape, map or crawl, so the search is the only call to move.

Set it up

1. In your SpecStream workspace, open Agents & API, choose API keys, then Create API key, and set a total limit. New accounts start with $10 of credit, no card.
2. Put the two lines in .env.local in the repository root, next to OPENAI_KEY. npm start runs tsx --env-file=.env.local src/run.ts, so that is the file it reads. With Docker, docker-compose.yml reads the same file through env_file.
3. Run npm start, type the question, and answer the breadth and depth prompts. The report lands in report.md, with the pages the searches found listed under its sources.

To check the address and the key before a run, send the same call the SDK sends. It makes one search on your key:

curl -s https://api.specstream.ai/firecrawl/v1/search \
  -H "Authorization: Bearer $SPECSTREAM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "open source deep research agents", "limit": 5, "scrapeOptions": {"formats": ["markdown"]}}'

What comes back

The reply is in Firecrawl's v1 shape, so processSerpResult reads it as it is:

Field the code reads	What SpecStream puts there
data[].url	The result's address
data[].markdown	The part of the page closest to the query, up to about 3,500 characters
data[].title, description	The result's title and snippet
data[].metadata	title, sourceURL, and publishedTime when the page has a date

The code keeps each result's markdown and trims it to 25,000 tokens before the model reads it. SpecStream sends the passages of the page closest to the query, not the whole page, so the model reads less text for each result.

How many searches a run makes, and the price

A run searches in levels. The first level writes up to breadth queries. Each query then starts a level of its own with half the breadth, rounded up, until depth runs out. That gives these counts at most:

How you start it	Defaults	Searches at most
npm start	breadth 4, depth 2	4 + 8 = 12
npm run api, POST /api/research	breadth 3, depth 3	3 + 6 + 6 = 15

Each Firecrawl search on the SpecStream address is one Search API call on the fast tier: $0.001 for up to 10 results, $1 per 1,000. The code asks for 5 results, so every call is the base price, and the page text comes with it at no extra charge. A default CLI run costs at most $0.012 in searches and a default API run at most $0.015. A call that returns no results costs nothing. The model calls stay on your OpenAI, Fireworks or other model bill. Every call comes out of your balance with no quote and no approval, and each one is listed on the Search API screen of your workspace.

The same key reaches the Search API directly, at $5 per 1,000 searches on the default tier, and the Extract API at $0.80 per 1,000 pages, where a page with no text is not charged. On accuracy, the Search API scores 97.9 F-score on all 1,000 SimpleQA Verified questions and 64.0 item F1 on all 200 WideSearch tasks (benchmarks).

Concurrency: the variable the code reads

The README says to raise CONCURRENCY_LIMIT on a paid plan. The code reads a different name:

const ConcurrencyLimit = Number(process.env.FIRECRAWL_CONCURRENCY) || 2;

To run more searches at once, set FIRECRAWL_CONCURRENCY in .env.local. CONCURRENCY_LIMIT changes nothing.

Going back

Delete FIRECRAWL_BASE_URL and put your Firecrawl key back in FIRECRAWL_KEY. The SDK then calls https://api.firecrawl.dev again. To stop the SpecStream key working everywhere, revoke it in Agents & API.

If what you need from the run is a list rather than a report, such as every company or paper that meets your conditions with a source on each value, compare deep research APIs first.