Extract API

POST /v1/extract returns the main text of up to 20 web pages or documents as markdown or plain text.

Make a request

Send 1 to 20 http or https URLs in urls. format is markdown (the default: headings, lists, and links kept) or text (plain text). The same URL twice is read once. Authentication

Terminal
curl https://api.specstream.ai/v1/extract \
  -H "Authorization: Bearer $SPECSTREAM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://en.wikipedia.org/wiki/Narbonne"], "format": "markdown"}'

Python
import os
import httpx

reply = httpx.post(
    "https://api.specstream.ai/v1/extract",
    headers={"Authorization": f"Bearer {os.environ['SPECSTREAM_API_KEY']}"},
    json={"urls": ["https://en.wikipedia.org/wiki/Narbonne"]},
    timeout=60,
)
print(reply.json()["data"]["results"][0]["text"])

typescript
const reply = await fetch("https://api.specstream.ai/v1/extract", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.SPECSTREAM_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ urls: ["https://en.wikipedia.org/wiki/Narbonne"] }),
})
const { data } = await reply.json()
console.log(data.results[0].text)

On MCP, call the read_pages tool with the same urls.

Price and retries

Each page read is $0.80 per 1,000 pages, from your balance. A page that gives no text is not charged. It needs no quote and no approval. Send an Idempotency-Key header (at most 128 characters) to retry safely: a call is charged once per key, and the same key again gets 409. Without one, SpecStream gives the call an ID and returns it as request_id.

The reply

Over REST the reply sits under data: results, one for each URL in the order sent, then request_id, took_ms, and price_usd, the price of the pages read. This is a real one, cut short:

JSON
{
  "data": {
    "results": [
      {
        "url": "https://en.wikipedia.org/wiki/Narbonne",
        "status": "ok",
        "title": "Narbonne - Wikipedia",
        "published": "2002-10-11T13:08:41+00:00",
        "read_from": "live",
        "text": "... ## Etymology\n\nThe etymology of the town's original name, **Narbo**, is lost in antiquity ..."
      }
    ],
    "took_ms": 441,
    "price_usd": 0.0008
  }
}

Field	What it holds
`url`	The URL, in the order sent.
`status`	ok (read, and charged), blocked (the site refused the read, or its robots.txt asks SpecStream not to read it), not_found (404 or 410), empty (no main text), unsupported (not a web page, text, or a document SpecStream reads: PDF, Word, Excel, EPUB, or an image), invalid_url (not a public http or https URL), or failed (no answer in time, or another error).
`text`	The page’s main text, without its menus, footers, and ads.
`title, published`	The page’s title, and the date it says it was published, when it says one.
`read_from`	live (the page now), site (the same page from the site’s own API or feed), or archive (its newest public web archive capture). For archive, captured_at says when the capture was taken.

How a page is read

Each URL gets up to 30 seconds, all URLs at once. A page that refuses a plain read is tried again as a browser would ask for it, then in a real browser, then from the site’s own feed, then from its newest public archive capture. The web fetcher says how a site opts out.

When a call stops

Status	What happened
`400`	urls is empty or has more than 20.
`401`	No key or token.
`402`	The balance can’t pay for every URL sent, or the call would go over the account’s monthly spending limit.
`403`	The call would go over the limit delegated to the account or the key, or the key has no finds:spend.
`409`	The Idempotency-Key was already charged. Send a new one.
`422`	The Idempotency-Key is over 128 characters.
`503`	The account used its calls for the minute. Wait the seconds in Retry-After and send it again. Nothing is charged.