Vision Agents

Give a Vision Agents voice or video agent live web search. Add SpecStream's MCP server with MCPServerRemote, and the agent calls it mid call, gets one answer with the pages that print it, and says it back. Or call the Search API from a registered function.

SpecStream for Vision AgentsAgent frameworks & builders

What you need

Python 3.10 to 3.13 and a Vision Agents project, such as one made with uvx vision-agents init, with your Stream keys and an LLM that supports function calling.
A SpecStream account. New accounts start with $10 of credit, no card. Create one.
A SpecStream API key with a spending limit.

1
1. Create an API key
In your SpecStream workspace, open Agents & API, choose API keys, then Create API key, and set a spending limit. You paste the key into the agent's .env in the next step.

2. Add the key to .env

Vision Agents projects load .env with load_dotenv(). Add the key next to your Stream and model keys:

.env
SPECSTREAM_API_KEY=YOUR_SPECSTREAM_API_KEY

3. Add SpecStream as a remote MCP server

MCPServerRemote connects over Streamable HTTP and sends the key as a header. Pass it in the agent's mcp_servers. When the agent joins a call, Vision Agents lists the server's tools and registers each one with the LLM. This is the agent.py that vision-agents init writes, with SpecStream added:

agent.py
import os

from dotenv import load_dotenv
from vision_agents.core import Agent, Runner, User
from vision_agents.core.agents import AgentLauncher
from vision_agents.core.mcp import MCPServerRemote
from vision_agents.plugins import gemini, getstream

load_dotenv()

INSTRUCTIONS = (
    "You're a helpful voice AI assistant. Keep responses short and conversational. "
    "For anything recent or factual, call mcp_0_search_web with the question in plain words, "
    "then say the answer in one or two sentences and name the site it came from. "
    "Call mcp_0_read_pages only when the user asks what a page says."
)


async def create_agent(**kwargs) -> Agent:
    specstream = MCPServerRemote(
        url="https://api.specstream.ai/mcp",
        headers={"Authorization": f"Bearer {os.environ['SPECSTREAM_API_KEY']}"},
        timeout=30.0,
        session_timeout=300.0,
    )
    return Agent(
        edge=getstream.Edge(),
        agent_user=User(name="My AI assistant", id="agent"),
        instructions=INSTRUCTIONS,
        llm=gemini.Realtime(),
        mcp_servers=[specstream],
    )


async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None:
    call = await agent.create_call(call_type, call_id)
    async with agent.join(call):
        await agent.simple_response(text="Say hi and introduce yourself.")
        await agent.finish()


runner = Runner(AgentLauncher(create_agent=create_agent, join_call=join_call))

if __name__ == "__main__":
    runner.cli()

4. Run it and ask

Start the agent, open the join link it prints, and ask a question about this week:

Terminal

uv run agent.py run

Tool names

Vision Agents registers each MCP tool as mcp_<server index>_<tool name>, so a server does not clash with another or with your own functions. With SpecStream first in mcp_servers, search is mcp_0_search_web and page reading mcp_0_read_pages. Name them in the instructions, as above, so the model calls them. The ToolStartEvent and ToolEndEvent events show each call with its arguments and time.

What comes back

search_web takes {"q": "<the question>"}. The reply holds answer, the value as the pages print it, with the page it was read from, and sources, the results read in rank order with their url, title, site and snippet. When no page prints a value that answers the question, answer is null and the sources are still there, so the agent can say what it found instead of guessing.

read_pages takes {"urls": ["<url>", ...]} and returns each page's text as Markdown, for a follow up such as "what does that article say about prices".

The server also lists the Deep Search tools, which build tables of rows. They run only through a quote with an approved maximum, and the key's spending limit holds on every call.

Or register a function that calls the Search API

To skip MCP, register an async function on the LLM. Vision Agents takes only async functions. The call waits without holding up the audio, and a failed search returns a sentence the agent can say instead of raising. Install httpx, then pass this llm to the agent with your STT and TTS:

Keep spoken answers short

A search answers in a few seconds. Tell the agent in its instructions to say a short line such as "Let me check" before it calls the tool, and to read back the answer and one source, not the whole list. Quoted phrases and site: in the question hold on every result, which helps when the user names a source.

Cost

Each search is one Search API call at $5 per 1,000 searches of up to 10 results, from your balance, with no quote and no approval. A search that returns no results costs nothing. Each page read_pages reads is $0.80 per 1,000 pages, and a page with no text is not charged. The reply's price_usd says what each call cost. Search API

Limits on the key

The key's spending limit holds on every call the agent makes, so a long call or a loop cannot spend past it. Give the agent server its own key, and revoke it in Agents & API to stop it at once. Every search is kept in your account's history with its answer and sources.

Remove

Take SpecStream out of mcp_servers, or remove the registered function, and revoke the key in Agents & API. Every tool on the server is described in the agent guide. Questions go to [email protected].

SpecStream for Vision Agents covers what it does in Vision Agents on one page. Every other tool has its guide in Connect your tools. You see the most a request can cost before it starts, and the matches are paid from your balance. Billing has the details. For help, email [email protected].