Skip to the page
Get started free

Open Deep Research

Run nickscamara's Open Deep Research on SpecStream in place of Firecrawl: search through the Firecrawl address with one added line, and read pages with the Extract API. Firecrawl has deprecated the extract endpoint the app is built on, and the app's search, scrape and page reading keep working on one SpecStream key.

SpecStream for Open Deep ResearchAI assistants

What you need

  • Open Deep Research from github.com/nickscamara/open-deep-research, run with pnpm dev, Docker Compose or on Vercel, with its model key (OPENAI_API_KEY or OPENROUTER_API_KEY) set.
  • A SpecStream key. Get your key on the Search API screen of your SpecStream workspace. New accounts start with $10 of credit, no card.

  1. 1. Put the key and the address in .env

    In the .env file at the root of the repository, put your SpecStream key in FIRECRAWL_API_KEY, the variable .env.example names, and add FIRECRAWL_API_URL with SpecStream's Firecrawl address. On Vercel, set the same two variables under the project's environment variables. With Docker Compose, the app service reads .env through env_file.

    .env
    FIRECRAWL_API_KEY=YOUR_SPECSTREAM_KEY
    FIRECRAWL_API_URL=https://api.specstream.ai/firecrawl

  2. 2. Point the Firecrawl client at the address

    The app builds one Firecrawl client in app/(chat)/api/chat/route.ts, with the key and no address, so the SDK calls https://api.firecrawl.dev. Add apiUrl, and the search and scrape tools and the search step of a deep research go to SpecStream. The pinned SDK, @mendable/firecrawl-js 1.15.7, adds /v1/search and /v1/scrape to the address, and SpecStream answers both in Firecrawl's v1 JSON.

    app/(chat)/api/chat/route.ts
    const app = new FirecrawlApp({
      apiKey: process.env.FIRECRAWL_API_KEY || '',
      apiUrl: process.env.FIRECRAWL_API_URL,
    });

  3. 3. Read pages with the Extract API

    The extract tool and the extract step of a deep research call app.extract, Firecrawl's /v1/extract, which SpecStream does not answer. Add this function under the client in the same file. It sends up to 20 URLs to the Extract API and returns each page's text in the shape both places already read: success, and data with url and data for each page.

    app/(chat)/api/chat/route.ts
    async function readPages(urls: string[]) {
      const reply = await fetch('https://api.specstream.ai/v1/extract', {
        method: 'POST',
        headers: {
          Authorization: `Bearer ${process.env.FIRECRAWL_API_KEY}`,
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({ urls: urls.slice(0, 20), format: 'markdown' }),
      });
      if (!reply.ok) {
        return { success: false, error: `Extract API answered ${reply.status}`, data: [] };
      }
      const { data } = await reply.json();
      return {
        success: true,
        data: data.results
          .filter((page: any) => page.status === 'ok')
          .map((page: any) => ({ url: page.url, data: page.text })),
      };
    }

  4. 4. Call it in the two places

    In the extract tool, replace app.extract(urls, { prompt }) with readPages(urls). In extractFromUrls, inside the deepResearch tool, replace the app.extract([url], { ... }) call and its prompt with readPages([url]). The rest of the file stays: extractFromUrls already keeps each item's data as a finding with its URL as the source.

    app/(chat)/api/chat/route.ts
    // extract tool
    const scrapeResult = await readPages(urls);
    

    // extractFromUrls, in deepResearch
    const result = await readPages([url]);

  5. 5. Restart and run a deep research

    Restart pnpm dev, or run docker compose up --build again. In a chat, choose the Deep Research tab under the message box and ask a question. The activity panel shows each search and each page read, and the sources list fills with the pages the searches found.

What changes and what stays

Searches go to https://api.specstream.ai/firecrawl/v1/search and single page scrapes to /v1/scrape, through the Firecrawl SDK the app already uses. Page reading for the extract tool and for deep research goes to POST https://api.specstream.ai/v1/extract. The models, the database, the rate limiter and the interface stay as they are.

Firecrawl's extract ran a model on each page with the prompt the app wrote. The Extract API returns the page's main text, without its menus, footers and ads, and the reasoning model reads that text in the analysis step. The prompt argument of the extract tool is no longer sent anywhere.

Why the extract calls need a change

Firecrawl's own write-up of this app now opens with a note dated October 29, 2026: Firecrawl has deprecated its extract endpoint, and Firecrawl /agent replaces it. Every finding in a deep research comes from that call in this app, so the change in the two places above is what keeps deep research working, on SpecStream or anywhere else.

What a deep research costs

A deep research runs at most 7 levels in at most 4 minutes 30 seconds. Each level makes one search and reads up to 4 pages: the URL the analysis asked for and the top 3 results. That is at most 7 searches and 28 page reads.

A search through the Firecrawl address runs SpecStream's fast search, at $1 per 1,000 searches of up to 10 results. The app asks for Firecrawl's default of 5. A page read costs $0.80 per 1,000 pages, and a page that gives no text is not charged. A full deep research costs less than 3 cents on SpecStream; the model calls stay on your model provider's bill.

Calls come out of your balance with no quote and no approval, and each one is listed on the Search API screen of your workspace. The same key reaches the Search API at $5 per 1,000 searches on the default tier, which scores 97.9 F-score on all 1,000 SimpleQA Verified questions (benchmarks).

Check the address before a run

This is the call the SDK sends for a search. It makes one search on your key:

Limits and errors

  • A 401 from either address means the key in FIRECRAWL_API_KEY is wrong, or was revoked.
  • A 402 from the Extract API means the balance can't pay for the pages sent. Add balance in the workspace.
  • When the account has used its calls for the minute, SpecStream answers 429 or 503 with Retry-After, and nothing is charged. The search step counts a failed search and stops after its set number of failures.
  • A page the site refuses comes back with the status blocked and is left out of data, so the app reads only pages that gave text.

Go back to Firecrawl

Remove FIRECRAWL_API_URL from .env and put your Firecrawl key back in FIRECRAWL_API_KEY. The client then calls https://api.firecrawl.dev again. To stop the SpecStream key working everywhere, revoke it under Agents & API in your workspace. Questions go to [email protected].

More

SpecStream for Open Deep Research covers what it does in Open Deep Research on one page. Every other tool has its guide in Connect your tools. You see the most a request can cost before it starts, and the matches are paid from your balance. Billing has the details. For help, email [email protected].