> SpecStream is the live web data engine for teams and agents. This is the Markdown copy of https://specstream.ai/docs/integrations/open-deep-research. Every page, with what SpecStream does and costs: https://specstream.ai/llms.txt

# Open Deep Research

Run nickscamara's Open Deep Research on SpecStream in place of Firecrawl: search through the Firecrawl address with one added line, and read pages with the Extract API. Firecrawl has deprecated the extract endpoint the app is built on, and the app's search, scrape and page reading keep working on one SpecStream key.

[SpecStream for Open Deep ResearchAI assistants](https://specstream.ai/integrations/open-deep-research.md)

## What you need

-   Open Deep Research from `github.com/nickscamara/open-deep-research`, run with `pnpm dev`, Docker Compose or on Vercel, with its model key (`OPENAI_API_KEY` or `OPENROUTER_API_KEY`) set.
-   A SpecStream key. [Get your key](https://specstream.ai/workspace/search?route=firecrawl&utm_source=open-deep-research&utm_medium=howto) on the Search API screen of your SpecStream workspace. New accounts start with $10 of credit, no card.

1.  1

    ## 1. Put the key and the address in .env

    In the `.env` file at the root of the repository, put your SpecStream key in `FIRECRAWL_API_KEY`, the variable `.env.example` names, and add `FIRECRAWL_API_URL` with SpecStream's Firecrawl address. On Vercel, set the same two variables under the project's environment variables. With Docker Compose, the `app` service reads `.env` through `env_file`.

    .env

    ```bash
    FIRECRAWL_API_KEY=YOUR_SPECSTREAM_KEY
    FIRECRAWL_API_URL=https://api.specstream.ai/firecrawl
    ```

2.  2

    ## 2. Point the Firecrawl client at the address

    The app builds one Firecrawl client in `app/(chat)/api/chat/route.ts`, with the key and no address, so the SDK calls `https://api.firecrawl.dev`. Add `apiUrl`, and the `search` and `scrape` tools and the search step of a deep research go to SpecStream. The pinned SDK, `@mendable/firecrawl-js` 1.15.7, adds `/v1/search` and `/v1/scrape` to the address, and SpecStream answers both in Firecrawl's v1 JSON.

    app/(chat)/api/chat/route.ts

    ```typescript
    const app = new FirecrawlApp({
      apiKey: process.env.FIRECRAWL_API_KEY || '',
      apiUrl: process.env.FIRECRAWL_API_URL,
    });
    ```

3.  3

    ## 3. Read pages with the Extract API

    The `extract` tool and the extract step of a deep research call `app.extract`, Firecrawl's `/v1/extract`, which SpecStream does not answer. Add this function under the client in the same file. It sends up to 20 URLs to the Extract API and returns each page's text in the shape both places already read: `success`, and `data` with `url` and `data` for each page.

    app/(chat)/api/chat/route.ts

    ```typescript
    async function readPages(urls: string[]) {
      const reply = await fetch('https://api.specstream.ai/v1/extract', {
        method: 'POST',
        headers: {
          Authorization: `Bearer ${process.env.FIRECRAWL_API_KEY}`,
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({ urls: urls.slice(0, 20), format: 'markdown' }),
      });
      if (!reply.ok) {
        return { success: false, error: `Extract API answered ${reply.status}`, data: [] };
      }
      const { data } = await reply.json();
      return {
        success: true,
        data: data.results
          .filter((page: any) => page.status === 'ok')
          .map((page: any) => ({ url: page.url, data: page.text })),
      };
    }
    ```

4.  4

    ## 4. Call it in the two places

    In the `extract` tool, replace `app.extract(urls, { prompt })` with `readPages(urls)`. In `extractFromUrls`, inside the `deepResearch` tool, replace the `app.extract([url], { ... })` call and its prompt with `readPages([url])`. The rest of the file stays: `extractFromUrls` already keeps each item's `data` as a finding with its URL as the source.

    app/(chat)/api/chat/route.ts

    ```typescript
    // extract tool
    const scrapeResult = await readPages(urls);
    
    // extractFromUrls, in deepResearch
    const result = await readPages([url]);
    ```

5.  5

    ## 5. Restart and run a deep research

    Restart `pnpm dev`, or run `docker compose up --build` again. In a chat, choose the **Deep Research** tab under the message box and ask a question. The activity panel shows each search and each page read, and the sources list fills with the pages the searches found.

## What changes and what stays

Searches go to `https://api.specstream.ai/firecrawl/v1/search` and single page scrapes to `/v1/scrape`, through the Firecrawl SDK the app already uses. Page reading for the `extract` tool and for deep research goes to `POST https://api.specstream.ai/v1/extract`. The models, the database, the rate limiter and the interface stay as they are.

Firecrawl's extract ran a model on each page with the prompt the app wrote. The Extract API returns the page's main text, without its menus, footers and ads, and the reasoning model reads that text in the analysis step. The `prompt` argument of the `extract` tool is no longer sent anywhere.

## Why the extract calls need a change

Firecrawl's own write-up of this app now opens with a note dated October 29, 2026: Firecrawl has deprecated its extract endpoint, and Firecrawl /agent replaces it. Every finding in a deep research comes from that call in this app, so the change in the two places above is what keeps deep research working, on SpecStream or anywhere else.

## What a deep research costs

A deep research runs at most 7 levels in at most 4 minutes 30 seconds. Each level makes one search and reads up to 4 pages: the URL the analysis asked for and the top 3 results. That is at most 7 searches and 28 page reads.

A search through the Firecrawl address runs SpecStream's fast search, at $1 per 1,000 searches of up to 10 results. The app asks for Firecrawl's default of 5. A page read costs $0.80 per 1,000 pages, and a page that gives no text is not charged. A full deep research costs less than 3 cents on SpecStream; the model calls stay on your model provider's bill.

Calls come out of your balance with no quote and no approval, and each one is listed on the Search API screen of your workspace. The same key reaches the [Search API](https://specstream.ai/search-api.md) at $5 per 1,000 searches on the default tier, which scores 97.9 F-score on all 1,000 SimpleQA Verified questions ([benchmarks](https://specstream.ai/benchmarks.md)).

## Check the address before a run

This is the call the SDK sends for a search. It makes one search on your key:

## Limits and errors

-   A `401` from either address means the key in `FIRECRAWL_API_KEY` is wrong, or was revoked.
-   A `402` from the Extract API means the balance can't pay for the pages sent. Add balance in the workspace.
-   When the account has used its calls for the minute, SpecStream answers `429` or `503` with `Retry-After`, and nothing is charged. The search step counts a failed search and stops after its set number of failures.
-   A page the site refuses comes back with the status `blocked` and is left out of `data`, so the app reads only pages that gave text.

## Go back to Firecrawl

Remove `FIRECRAWL_API_URL` from `.env` and put your Firecrawl key back in `FIRECRAWL_API_KEY`. The client then calls `https://api.firecrawl.dev` again. To stop the SpecStream key working everywhere, revoke it under **Agents & API** in your workspace. Questions go to [support@specstream.ai](mailto:support@specstream.ai).

## More

[SpecStream for Open Deep Research](https://specstream.ai/integrations/open-deep-research.md) covers what it does in Open Deep Research on one page. Every other tool has its guide in [Connect your tools](https://specstream.ai/docs/integrations.md). You see the most a request can cost before it starts, and the matches are paid from your balance. [Billing](https://specstream.ai/docs/billing.md) has the details. For help, email [support@specstream.ai](mailto:support@specstream.ai).