Khoj web search

The environment before and after

Khoj reads KHOJ_SEARXNG_URL from the server's environment and asks SearXNG for its HTML results page. SpecStream answers format=html with SearXNG's own markup at https://api.specstream.ai/searxng/<key>, with the key as a part of the address, so Khoj's parser reads it as it is.

# docker-compose.yml, server environment, before
      - KHOJ_SEARXNG_URL=http://search:8080

# after: no search container
      - KHOJ_SEARXNG_URL=https://api.specstream.ai/searxng/YOUR_SPECSTREAM_KEY

Khoj tries Serper, Exa, Firecrawl and Google before SearXNG, so leave their keys unset. The search service can go from docker-compose.yml once search answers.

curl "https://api.specstream.ai/searxng/$SPECSTREAM_API_KEY/search?q=latest+Khoj+release&format=html"

Search and read pages with the Exa variables

Set EXA_API_URL and EXA_API_KEY and Khoj sends its searches to /search on that address with the key in x-api-key. With no WebScraper row in Django admin, the same two variables set Khoj's Exa web scraper, which reads pages at /contents. Both run on SpecStream. The Exa migration guide and the Exa contents guide list every field.

# before
      - EXA_API_KEY=your_exa_api_key

# after
      - EXA_API_KEY=YOUR_SPECSTREAM_KEY
      - EXA_API_URL=https://api.specstream.ai/exa

curl -X POST https://api.specstream.ai/exa/search \
  -H "x-api-key: $SPECSTREAM_API_KEY" -H "Content-Type: application/json" \
  -d '{"query": "latest Khoj release", "numResults": 10}'

A WebScraper row in Django admin

When the admin panel already has WebScraper rows, Khoj uses them and not the variables. EXA_API_URL only fills in a new row. Open the Exa row, set api_url to https://api.specstream.ai/exa and the API key to your SpecStream key, and save.

Already on Firecrawl? Change one URL

Set FIRECRAWL_API_URL to https://api.specstream.ai/firecrawl, with no version: Khoj adds /v2/search for a search and /v2/scrape for a page, and sends the key as a Bearer token. The Firecrawl migration guide lists every field.

# before
      - FIRECRAWL_API_KEY=fc-...

# after
      - FIRECRAWL_API_KEY=YOUR_SPECSTREAM_KEY
      - FIRECRAWL_API_URL=https://api.specstream.ai/firecrawl

What stays the same

Khoj's own search and scraper code reads each reply, unchanged. The SearXNG route returns SearXNG's results page, 10 results a search. The Exa route returns results[] with url, title, publishedDate and text, and the Firecrawl route data.web[]. Every parameter is in the SearXNG migration guide.

$0.005 a search and $0.0008 a page

A search of up to 10 results costs $0.005, which is $5 per 1,000, with no daily limit. Each page read through the Firecrawl scrape or Exa contents route costs $0.0008, which is $0.80 per 1,000 pages, and a page with no text costs nothing.

There is no monthly plan, no credit pack that expires and no reset on the first of the month. Credit you buy never expires.

The most accurate web search API in the world

On SimpleQA Verified the SpecStream Search API scores 97.9 F-score on all 1,000 official questions, 978 of them correct. On WideSearch it scores 64.0 item F1 and 43.4 row F1 on all 200 official tasks. The Extract API scores 0.908 F1 on WCXB. Every score and its method is on the benchmarks page.

About one second a search, measured under load

Every drop-in route runs on the fast search. On November 9, 2026 production took 1, 5, 10 and 20 searches a second through it and answered 718 of 720. The median was 0.82 s at 1 search a second and 1.07 s at 20 a second, and the 95th percentile stayed between 1.09 and 1.73 s. One account can send up to 26,666 searches a minute; a call over that gets a 429 with Retry-After and is not charged.

You decide what is kept, on each key

Give Khoj a key with zero_retention and a search made with it keeps no request text and no answer after the reply. It is not in the history and its log lines carry neither. The charge keeps the account, the key, the request ID, the price and the time. spend_cap_cents is the most the key can spend.

curl -X POST https://api.specstream.ai/v1/account/keys \
  -H "Authorization: Bearer $SPECSTREAM_API_KEY" -H "Content-Type: application/json" \
  -d '{"name": "production", "scopes": ["account:read", "finds:spend"], "spend_cap_cents": 500000, "zero_retention": true}'

Search, pages and rows on one key

The same key reads up to 20 pages or documents a call with the Extract API at $0.80 per 1,000 pages, and a page with no text is not charged. Deep Search takes a request in plain words and returns every match on the live web as structured rows, at 5¢, 15¢ or $1 a row, with a source URL on every value.

Serper

Search sold in credit packs: $1.00 per 1,000 in the pack of 50,000, with credits valid for 6 months and 2,500 free queries to start. Khoj fixes Serper's address in its code and tries it first. SpecStream searches the live web at $5 per 1,000 on credit that never expires, and reads the pages too.

Exa

Search at $7 per 1,000 requests on auto and $4 on lite, and page contents at $1 per 1,000 pages per content type. Set EXA_API_URL to https://api.specstream.ai/exa and the same calls cost $0.005 a search and $0.0008 a page.

Firecrawl

Search at 2 credits per 10 results, on a free plan of 500 searches or 1,000 pages and paid plans from $19 a month for Hobby. Set FIRECRAWL_API_URL to https://api.specstream.ai/firecrawl and search and page reads run on SpecStream, on one key.

SearXNG, self-hosted

The search container in Khoj's docker-compose.yml. An engine that answers too many requests is suspended for 3,600 seconds, and one that shows a CAPTCHA for 86,400. SpecStream answers the same HTML call at $0.005 a search, with no container to run.

Google Custom Search

Khoj fixes Google's address at https://www.googleapis.com/customsearch/v1, and Google gives existing Custom Search JSON API customers until January 1, 2027 to move. SpecStream answers through KHOJ_SEARXNG_URL or EXA_API_URL today, with 97.9 F-score on all 1,000 SimpleQA Verified questions.