Skip to the page

SpecStream’s web fetcher

This page is for website owners. It explains when SpecStream visits your site, how to recognize its requests, and how to ask it to stop.

What it does

SpecStream fetches public web pages when a customer runs a request, for example “dental clinics in Mumbai with online booking”. It opens the pages that can confirm or rule out each result and reads them at that moment.

  • Every fetch comes from a customer’s request. SpecStream doesn’t crawl the web on its own.
  • It doesn’t build a search index from the pages it reads.
  • It doesn’t train models on the pages it reads.
  • It only reads public pages. It doesn’t sign in to your site.
  • It limits how many requests it sends to one site at a time.

How to recognize it

Signed requests from SpecStream carry this user agent:

Mozilla/5.0 (compatible; SPECSTREAM/1.0; +https://specstream.ai)

They are signed with HTTP message signatures (Web Bot Auth, RFC 9421). Each signed request has a Signature-Agent: "https://specstream.ai" header, and you can check the signature against the public key in our key directory:

https://specstream.ai/.well-known/http-message-signatures-directory

Opting out

To stop SpecStream using your website in results, submit it on the opt-out page. This applies to future requests and to results people have already saved. For anything else, contact us.