<!-- twin of https://specstream.ai/web-search/llamaindex -->
# LlamaIndex FireCrawlWebReader on SpecStream: one argument

Compared on Nov 10, 2026. The page: https://specstream.ai/web-search/llamaindex

LlamaIndex's `FireCrawlWebReader` has an `api_url` argument. Set it to `https://api.specstream.ai/firecrawl`, pass a SpecStream key as `api_key`, and the reader's scrape, crawl and map modes read through SpecStream.

The reader sends the same Firecrawl v2 calls and builds the same LlamaIndex `Document` objects. Your index, your node parser and your query engine stay as they are.

A page read costs $0.80 per 1,000, and a page with no text is not charged. The Extract API scores 0.908 F1 on WCXB. A new account gets $10 of credit, no card.

[Get an API key](https://specstream.ai/signup?src=alternatives-web-search-llamaindex&utm_source=alternatives-md&utm_campaign=web-search-llamaindex&returnTo=%2Fworkspace%2Fagents%3Ftab%3Dkeys) New accounts get $10 of credit, no card. It covers 2,000 searches.

## FireCrawlWebReader on SpecStream and on Firecrawl, from each company's own pages and the reader's source on Nov 10, 2026

| | SpecStream | Firecrawl |
|---|---|---|
| FireCrawlWebReader arguments | `api_url="https://api.specstream.ai/firecrawl"`, `api_key` your SpecStream key | `api_url` left out, so firecrawl-py calls Firecrawl's own address, `api_key` a Firecrawl key |
| Modes | `scrape`, `crawl` and `map`, each run through the reader on November 10, 2026 | The reader's docstring lists `scrape`, `crawl`, `map`, `search` and `extract`. `crawl` is the default |
| Scrape | One page read, $0.0008 | 1 credit a page |
| Crawl | Each crawled page with text is one page read, $0.0008 | 1 credit a page |
| Map | One page read, $0.0008, and one `Document` per link | 1 credit a call |
| Price | $0.80 per 1,000 pages, with no monthly plan. Credit you buy never expires | Hobby $19 a month for 5,000 credits. Standard $99 a month for 100,000 |
| Failed page | A page that gives no text is not charged | A page that returns a document is billed. No document is 0 credits |
| Free to start | $10 of credit, no card, which covers 12,500 pages | 1,000 free credits a month, no card |

## Why LlamaIndex pipelines read the web through SpecStream

### The change is one argument

`FireCrawlWebReader` passes `api_url` to the Firecrawl client when it is set, and the client calls that address in place of Firecrawl's. Before:

```
from llama_index.readers.web import FireCrawlWebReader

reader = FireCrawlWebReader(
    api_key="fc-...",
    mode="crawl",
    params={"limit": 2},
)
documents = reader.load_data(url="https://example.com")
```

After, with the SpecStream key and one more argument:

```
import os
from llama_index.readers.web import FireCrawlWebReader

reader = FireCrawlWebReader(
    api_key=os.environ["SPECSTREAM_API_KEY"],
    api_url="https://api.specstream.ai/firecrawl",
    mode="crawl",
    params={"limit": 2},
)
documents = reader.load_data(url="https://example.com")
```

The same reader runs `mode="scrape"` for one page and `mode="map"` for a site's links. On November 10, 2026, llama-index-readers-web 0.7.0 with firecrawl-py 4.55.0 read https://example.com in scrape and crawl, and mapped https://docs.python.org/3/ into 44 documents, all on `https://api.specstream.ai/firecrawl`.

### The same Firecrawl calls underneath

The reader's scrape is a Firecrawl v2 scrape. SpecStream answers it in Firecrawl's shape, with `data.markdown` and `data.metadata`, so the reader's `Document` gets the page text and its metadata. The same call in plain Python:

```
import os, requests

reply = requests.post(
    "https://api.specstream.ai/firecrawl/v2/scrape",
    headers={"Authorization": f"Bearer {os.environ['SPECSTREAM_API_KEY']}"},
    json={"url": "https://example.com"},
).json()

print(reply["data"]["metadata"])
print(reply["data"]["markdown"][:500])
```

Every parameter and field of the scrape route is in the [Firecrawl guide](https://specstream.ai/migrate/firecrawl?utm_source=alternatives&utm_campaign=web-search-llamaindex), and the map and crawl routes are in the [Firecrawl crawl and map guide](https://specstream.ai/migrate/firecrawl-crawl?utm_source=alternatives&utm_campaign=web-search-llamaindex).

### $0.80 per 1,000 pages, and empty pages are free

A scraped page is one page read at $0.0008. A crawl charges one page read for each page with text, and a map is one page read. A page that gives no text is not charged. There is no monthly plan, credit you buy never expires, and a new account starts with $10 of credit, which covers 12,500 pages. A busy call gets a `429` with `Retry-After` and charges nothing.

### The text your index gets is the text on the page

The Extract API, which answers every page the reader asks for, scores 0.908 F1 on WCXB. The Search API scores 97.9 F-score on all 1,000 SimpleQA Verified questions, 978 of them correct, and 64.0 item F1 and 43.4 row F1 on all 200 WideSearch tasks. Every score and its method is on the [benchmarks](https://specstream.ai/benchmarks?utm_source=alternatives&utm_campaign=web-search-llamaindex) page.

### You decide what is kept, on each key

Create the key with `zero_retention` and a call made with it keeps no request text and no answer after the reply. `spend_cap_cents` is the most the key can spend, so a crawl in a loop stops at your number.

```
curl -X POST https://api.specstream.ai/v1/account/keys \
  -H "Authorization: Bearer $SPECSTREAM_API_KEY" -H "Content-Type: application/json" \
  -d '{"name": "production", "scopes": ["account:read", "finds:spend"], "spend_cap_cents": 500000, "zero_retention": true}'
```

### Search, pages and rows on one key

The same key searches the live web with the [Search API](https://specstream.ai/web-search?utm_source=alternatives&utm_campaign=web-search-llamaindex) at $5 per 1,000 searches of up to 10 results, with no daily limit, and reads up to 20 pages or documents a call with the [Extract API](https://specstream.ai/extract-api?utm_source=alternatives&utm_campaign=web-search-llamaindex). [Deep Search](https://specstream.ai/deep-search?utm_source=alternatives&utm_campaign=web-search-llamaindex) returns every match for a request in plain words as structured rows, at 5¢, 15¢ or $1 a row, with a source URL on every value. The most a run can cost is approved before it starts.

## Other ways to read pages into LlamaIndex

### Firecrawl

The reader's default. Scrape and crawl cost 1 credit a page and a map 1 credit a call, on plans from $19 a month for 5,000 credits to $99 a month for 100,000. SpecStream runs the same reader at $0.80 per 1,000 pages, with no plan, behind one argument.

### Exa

A search API that sells page contents at $1 per 1,000 pages. SpecStream reads pages at $0.80 per 1,000, 20 pages or documents to a call.

### Spider

A scrape and crawl API billed by bandwidth, from $1 per GB. SpecStream bills by the page, $0.0008 each, and a page with no text costs nothing.

## How to move FireCrawlWebReader to SpecStream

1. Create a key under Agents & API in the SpecStream workspace. A new account gets $10 of credit, no card, which covers 12,500 pages.

2. Check the key with one scrape. A `200` with `data.markdown` means the reader gets the same.

```
curl -X POST https://api.specstream.ai/firecrawl/v2/scrape \
  -H "Authorization: Bearer $SPECSTREAM_API_KEY" -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'
```

3. Add `api_url` to the reader and put the SpecStream key in `api_key`:

```
import os
from llama_index.readers.web import FireCrawlWebReader

reader = FireCrawlWebReader(
    api_key=os.environ["SPECSTREAM_API_KEY"],
    api_url="https://api.specstream.ai/firecrawl",
    mode="scrape",
)
documents = reader.load_data(url="https://example.com")
```

4. Run your pipeline. Each page read shows in your SpecStream request log with its price.

## Frequently asked questions

### Can LlamaIndex's Firecrawl reader use another API address?

Yes. `FireCrawlWebReader` takes `api_url`; set it to `https://api.specstream.ai/firecrawl` with a SpecStream key in `api_key`, and scrape, crawl and map run on SpecStream at $0.80 per 1,000 pages.

### What is FireCrawlWebReader api_url for?

It is the base address the Firecrawl client calls, and it is optional. Left out, the reader calls Firecrawl; set to `https://api.specstream.ai/firecrawl`, it reads through SpecStream.

### How do I crawl a website with LlamaIndex?

Use `FireCrawlWebReader` with `mode="crawl"` and a page limit in `params`, then `load_data(url=...)`. On SpecStream each crawled page with text costs $0.0008.

### What is the best Firecrawl alternative for LlamaIndex?

SpecStream, at $0.80 per 1,000 pages with 0.908 F1 on WCXB. It answers the reader's own Firecrawl calls, so one argument moves the pipeline and nothing else changes.

### How much does FireCrawlWebReader cost on SpecStream?

$0.0008 a page for scrape and crawl, and $0.0008 for a whole map. A page with no text is not charged, and $10 of credit, no card, covers 12,500 pages.

### Does the LlamaIndex reader still return Documents?

Yes. The reader builds its `Document` objects from the reply, and on November 10, 2026 it returned 1 document for a scrape and 44 for a map of https://docs.python.org/3/ on SpecStream.

## Sources

SpecStream wrote this page. The reader's arguments come from its source on GitHub, and Firecrawl's figures from its own pricing and billing pages.

- [LlamaIndex, firecrawl_web/base.py](https://raw.githubusercontent.com/run-llama/llama_index/main/llama-index-integrations/readers/llama-index-readers-web/llama_index/readers/web/firecrawl_web/base.py), read Nov 10, 2026
- [llama-index-readers-web, pyproject.toml](https://raw.githubusercontent.com/run-llama/llama_index/main/llama-index-integrations/readers/llama-index-readers-web/pyproject.toml), read Nov 10, 2026
- [Firecrawl pricing](https://www.firecrawl.dev/pricing), read Nov 10, 2026
- [Firecrawl billing](https://docs.firecrawl.dev/billing.md), read Nov 10, 2026
- [Exa pricing](https://exa.ai/docs/admin/pricing.md), read Nov 10, 2026
- [Spider pricing](https://spider.cloud/pricing), read Nov 10, 2026
- [SpecStream benchmarks](https://specstream.ai/benchmarks), read Nov 10, 2026
- [SpecStream pricing](https://specstream.ai/pricing), read Nov 10, 2026
- [SpecStream for agents](https://specstream.ai/agents.md), read Nov 10, 2026