Gemini Computer Use

Run Gemini computer use on a browser that SpecStream hosts. One call opens a session, Playwright connects to its connect_url, and Gemini drives the page: it reads a screenshot, answers with an action, and your loop carries it out. The browser costs $0.10 an hour, charged by the second, so a ten minute task costs under 2 cents of browser time. The script below is the whole loop in one Python file.

SpecStream for Gemini Computer UseAgent frameworks & builders

What you need

A Gemini API key from Google AI Studio (opens in a new tab), set as GEMINI_API_KEY, and Python with google-genai 2.7.0 or higher, Playwright and requests.
A SpecStream key. Get your key on the Search API screen of your SpecStream workspace. New accounts start with $10 of credit, no card.

1. Install the packages and set both keys

Google's guide also runs playwright install chromium. You can leave that out here: the browser runs at SpecStream, and Playwright on your machine only connects to it.

Terminal
pip install google-genai playwright requests

export GEMINI_API_KEY="your Gemini key"
export SPECSTREAM_API_KEY="your SpecStream key"

2
2. Open a browser session
POST /v1/browser/sessions opens the browser. Every field is optional. timeout_s sets the most seconds the session runs, from 60 to 3,600, and viewport sets the screen Gemini sees. Google's examples use 1440 by 900, so the script asks for the same.
The reply holds the session under data: its id, a connect_url that speaks the Chrome DevTools Protocol, and a live_url that shows the browser's screen while the agent works. The token in connect_url belongs to this session alone, so you can hand it to a sandbox without your API key.
Terminal
curl -X POST https://api.specstream.ai/v1/browser/sessions \ -H "Authorization: Bearer $SPECSTREAM_API_KEY" \ -H "Content-Type: application/json" \ -d '{"timeout_s": 600, "viewport": {"width": 1440, "height": 900}}'

3. Save the agent loop

Save this as gemini_agent.py. It opens the session, connects Playwright with connect_over_cdp, and sends Gemini the goal with a first screenshot through client.interactions.create and the computer_use tool in the browser environment.

Each turn, the script reads the function_call steps, carries out each action, and sends back one function_result per call with the page's URL and a new screenshot. previous_interaction_id continues the same task. The loop ends when Gemini answers with text and no action, or after MAX_TURNS turns.

Gemini gives coordinates on a 1000 by 1000 grid. px scales each one to the viewport before the click. act handles every action Google lists for the browser environment, including scroll, drag_and_drop, press_key and hotkey.

The finally block releases the session, so the charge stops that second even when the script fails, and prints what the session cost.

gemini_agent.py
import base64
import json
import os
import time

import requests
from google import genai
from playwright.sync_api import sync_playwright

SESSIONS = "https://api.specstream.ai/v1/browser/sessions"
HEADERS = {"Authorization": "Bearer " + os.environ["SPECSTREAM_API_KEY"]}
MODEL = "gemini-4-hydrogen"
WIDTH, HEIGHT = 1440, 900
TOOLS = [{"type": "computer_use", "environment": "browser"}]
GOAL = "Find the title of the top story on Hacker News and the site it links to."
START = "https://news.ycombinator.com"
MAX_TURNS = 20

KEYS = {
    "control": "Control", "ctrl": "Control", "cmd": "Meta", "command": "Meta", "shift": "Shift",
    "alt": "Alt", "enter": "Enter", "return": "Enter", "tab": "Tab", "escape": "Escape",
    "backspace": "Backspace", "delete": "Delete", "space": "Space", "pageup": "PageUp",
    "pagedown": "PageDown", "up": "ArrowUp", "down": "ArrowDown", "left": "ArrowLeft",
    "right": "ArrowRight",
}


def key(name):
    return KEYS.get(name.lower(), name)


def px(value, size):
    # Gemini answers on a 1000 by 1000 grid. Scale each coordinate to the viewport.
    return int(value / 1000 * size)


def act(page, name, args):
    x = px(args["x"], WIDTH) if "x" in args else None
    y = px(args["y"], HEIGHT) if "y" in args else None
    if name == "click":
        page.mouse.click(x, y)
    elif name == "double_click":
        page.mouse.dblclick(x, y)
    elif name == "triple_click":
        page.mouse.click(x, y, click_count=3)
    elif name == "right_click":
        page.mouse.click(x, y, button="right")
    elif name == "middle_click":
        page.mouse.click(x, y, button="middle")
    elif name == "move":
        page.mouse.move(x, y)
    elif name == "mouse_down":
        page.mouse.move(x, y)
        page.mouse.down()
    elif name == "mouse_up":
        page.mouse.move(x, y)
        page.mouse.up()
    elif name == "drag_and_drop":
        page.mouse.move(px(args["start_x"], WIDTH), px(args["start_y"], HEIGHT))
        page.mouse.down()
        page.mouse.move(px(args["end_x"], WIDTH), px(args["end_y"], HEIGHT), steps=10)
        page.mouse.up()
    elif name == "scroll":
        amount = int(args.get("magnitude_in_pixels", 300))
        moves = {"up": (0, -amount), "down": (0, amount), "left": (-amount, 0), "right": (amount, 0)}
        page.mouse.move(x, y)
        page.mouse.wheel(*moves[args["direction"]])
    elif name == "type":
        if x is not None and y is not None:
            page.mouse.click(x, y)
        page.keyboard.type(args["text"])
        if args.get("press_enter"):
            page.keyboard.press("Enter")
    elif name == "press_key":
        page.keyboard.press(key(args["key"]))
    elif name == "key_down":
        page.keyboard.down(key(args["key"]))
    elif name == "key_up":
        page.keyboard.up(key(args["key"]))
    elif name == "hotkey":
        page.keyboard.press("+".join(key(item) for item in args["keys"]))
    elif name == "navigate":
        page.goto(args["url"])
    elif name == "go_back":
        page.go_back()
    elif name == "go_forward":
        page.go_forward()
    elif name == "wait":
        time.sleep(args.get("seconds", 1))
    elif name != "take_screenshot":  # every reply carries a new screenshot
        raise ValueError("No handler for " + name)


def screenshot(page):
    return base64.b64encode(page.screenshot(type="png")).decode("utf-8")


def allowed(args):
    decision = args.get("safety_decision") or {}
    if decision.get("decision") != "require_confirmation":
        return False
    print("Gemini asks before this step:", decision.get("explanation", ""))
    if input("Allow it? (y/n) ").strip().lower() != "y":
        raise SystemExit("Stopped before the step.")
    return True


created = requests.post(
    SESSIONS,
    headers=HEADERS,
    json={"timeout_s": 600, "viewport": {"width": WIDTH, "height": HEIGHT}},
    timeout=60,
)
created.raise_for_status()
session = created.json()["data"]
print("Watch the browser:", session["live_url"])

client = genai.Client()
try:
    with sync_playwright() as playwright:
        browser = playwright.chromium.connect_over_cdp(session["connect_url"])
        page = browser.contexts[0].new_page()
        page.set_viewport_size({"width": WIDTH, "height": HEIGHT})
        page.goto(START)

        interaction = client.interactions.create(
            model=MODEL,
            input=[
                {"type": "text", "text": GOAL},
                {"type": "image", "data": screenshot(page), "mime_type": "image/png"},
            ],
            tools=TOOLS,
        )
        for turn in range(MAX_TURNS):
            calls = [step for step in interaction.steps if step.type == "function_call"]
            if not calls:
                print(" ".join(
                    block.text
                    for step in interaction.steps if step.type == "model_output"
                    for block in step.content if block.type == "text"
                ))
                break
            done = []
            for call in calls:
                args = dict(call.arguments)
                print("Step:", call.name, args.get("intent", ""))
                outcome = {"safety_acknowledgement": True} if allowed(args) else {}
                try:
                    act(page, call.name, args)
                    page.wait_for_load_state(timeout=5000)
                except Exception as error:
                    outcome["error"] = str(error)
                done.append((call.name, call.id, outcome))
            time.sleep(1)
            shot = screenshot(page)
            interaction = client.interactions.create(
                model=MODEL,
                previous_interaction_id=interaction.id,
                input=[
                    {
                        "type": "function_result",
                        "name": name,
                        "call_id": call_id,
                        "result": [
                            {"type": "text", "text": json.dumps({"url": page.url, **outcome})},
                            {"type": "image", "data": shot, "mime_type": "image/png"},
                        ],
                    }
                    for name, call_id, outcome in done
                ],
                tools=TOOLS,
            )
finally:
    ended = requests.post(SESSIONS + "/" + session["id"] + "/release", headers=HEADERS, timeout=60)
    print("The session cost $" + str(ended.json()["data"]["charged_usd"]))

4. Run it

Change GOAL and START to your task, then run the script. It prints the live_url first: open it to watch the browser. Each step prints the action and Gemini's intent, the reason it gives for that action. The last lines are Gemini's answer and the session's cost.

Terminal
python gemini_agent.py

Watch the browser: https://...
Step: click Open the top story to see where it links.
Step: go_back Return to the front page.
The top story is ...
The session cost $0.0021

When Gemini asks before a step

Gemini can attach a safety_decision to an action, for example before a payment, a sent message or a cookie consent. When the decision is require_confirmation, Google's rule is that the end user is asked. The script prints Gemini's explanation and waits for y. After a yes it carries out the step and sets safety_acknowledgement in the result, as Google's guide shows. After a no it stops, and the session is released.

In a product, replace the input call in allowed with your own approval screen. Keep the stop: an agent that skips the question can buy, send or accept something for your user.

Keep the agent on the sites you chose

allowed_domains on the session lists the hosts the browser may open, such as example.com or *.example.com. A page outside them is refused, whatever the model asks for.
timeout_s ends the session after that many seconds, connected or not, so a loop that never finishes cannot run all day. The most is 3,600.
MAX_TURNS ends the loop on your side. Google's sample stops at 5 turns. Most real tasks need more, so the script uses 20.
block_media set to true skips images, video, audio and fonts. Leave it off for computer use: Gemini reads the screenshot, and a page without images is harder to read.
To add Google's own check for hidden instructions in a page, add "enable_prompt_injection_detection": True to the tool in TOOLS.

Choose the model

The script uses gemini-4-hydrogen, the model in Google's examples. Google's guide also lists gemini-4-carbon, gemini-4-argon, gemini-3.8-flash, gemini-3.5-flash-lite and gemini-3-flash-preview for computer use. Change MODEL and nothing else. Code written for the older gemini-2.5-computer-use-preview-10-2025 used generate_content and other action names, so it does not run on this loop as it is.

If something doesn't work

What you see	What to do
Clicks land in the wrong place	The coordinates were not scaled, or the viewport differs from `WIDTH` and `HEIGHT`. Keep `px` on every coordinate and the same two numbers in the session's `viewport`.
`401` when the session opens	The SpecStream key is unknown, expired or revoked. Create a new key in your workspace and set `SPECSTREAM_API_KEY` again.
`402` when the session opens	The balance does not hold the first minute of the session. Add balance in your SpecStream workspace.
`422` when the session opens	A field is wrong, and the reply names it. `timeout_s` goes from 60 to 3,600.
Playwright cannot connect	The session has ended, so its `connect_url` no longer works. A session ends at its `timeout_s`, and at once when the client disconnects. Open a new session.
A step returns an `error` that names a blocked page	The host is outside `allowed_domains`, is a private address, or the site opted out. Gemini reads the error in the result and can choose another step.
`No handler for ...` in a result	Gemini called an action the script does not know, such as a custom function you added. Add a branch for it in `act`.
The loop ends without an answer	The task needed more than `MAX_TURNS` turns. Raise it, and raise `timeout_s` with it.

Cost

Browser time costs $0.10 an hour, charged by the second. Driven over connect_url with Playwright, as on this page, steps are not charged: a ten minute session costs $0.017 and the longest session, one hour, costs $0.10. The session's charged_usd shows the amount, and the script prints it. Sending one step at a time to the session's /actions route costs $1 per 1,000 steps on top of the time.

Google bills the Gemini tokens, and each turn sends one screenshot. The rates are on pricing.

When the task only needs to read

Computer use is for pages that need clicks: a form, a filter, a flow behind a button. For the text of a page, the Extract API is $0.80 per 1,000 pages, with a real browser where the page needs one. For an answer from the web, the Search API is $5 per 1,000 searches of up to 10 results. Both run on the same key, and the REST API reference lists every call. See also headless browsers for AI agents and the Browser Use guide.

Remove

Delete gemini_agent.py and revoke the key in your SpecStream workspace. A session that is still open ends at its timeout_s. To end it now, call POST /v1/browser/sessions/{id}/release. Questions: [email protected].

SpecStream for Gemini Computer Use covers what it does in Gemini Computer Use on one page. Every other tool has its guide in Connect your tools. You see the most a request can cost before it starts, and the matches are paid from your balance. Billing has the details. For help, email [email protected].