> SpecStream is the live web data engine for teams and agents. This is the Markdown copy of https://specstream.ai/docs/fetcher. Every page, with what SpecStream does and costs: https://specstream.ai/llms.txt

# SpecStream’s web fetcher

This page is for website owners. It explains when SpecStream visits your site, how to recognize its requests, and how to ask it to stop.

## What it does

SpecStream fetches public web pages when a customer runs a request, for example “dental clinics in Mumbai with online booking”. It opens the pages that hold the answer for each result and reads them at that moment.

-   Every fetch comes from a customer’s request. SpecStream doesn’t crawl the web on its own.
-   It doesn’t build a search index from the pages it reads.
-   It doesn’t train models on the pages it reads.
-   It only reads public pages. It doesn’t sign in to your site.
-   It limits how many requests it sends to one site at a time.

## How to recognize it

Signed requests from SpecStream carry this user agent:

```
Mozilla/5.0 (compatible; SPECSTREAM/1.0; +https://specstream.ai)
```

They are signed with HTTP message signatures (Web Bot Auth, RFC 9421). Each signed request has a `Signature-Agent: "https://specstream.ai"` header, and you can check the signature against the public key in our key directory:

```
https://specstream.ai/.well-known/http-message-signatures-directory
```

## Opting out

To stop SpecStream using your website in results, submit it on the [opt-out page](https://specstream.ai/opt-out.md). This applies to future requests and to results people have already saved. For anything else, [contact us](https://specstream.ai/talk-to-a-human.md).