Skip to the page

Neon

Write a Neon Function that searches the live web with SpecStream and saves every page it finds to a table in your Neon Postgres. Your app reads the results with plain SQL, and the SpecStream key stays inside the function.

SpecStream for Neon, Data & warehouses

What you need

  • A Neon project in a region that runs Functions: AWS US East (Ohio), AWS US East (N. Virginia), AWS Europe (Frankfurt) or AWS Asia Pacific (Singapore).
  • The latest neon CLI (npm install -g neon@latest), signed in, and Node.js 20 or later.
  • A SpecStream account. New accounts start with $10 of credit, no card. Create one.
  • A SpecStream API key with a spending limit, which step 1 creates.
  1. 1. Create an API key

    In your SpecStream workspace, open Agents & API, choose API keys, then Create API key. Turn on Authorize paid work: a key without it can't search or read pages. Set a Total spending limit above $0, because that limit is what caps the spending of everything that uses the key.

    Copy the key when it's shown. It isn't shown again.

  2. In a new directory, run neon link and pick your project and branch. It writes DATABASE_URL and the branch's other variables to a local .env or .env.local file. Install the packages the function uses:

    Terminal
    neon link
    npm install @neon/config @neon/functions hono pg
    npm install --save-dev @types/pg
  3. 3. Create the table

    Run this in the SQL Editor of the Neon Console, or with psql "$DATABASE_URL". One row per page and question:

    SQL Editor
    CREATE TABLE web_sources (
    query text NOT NULL,
    url text NOT NULL,
    title text,
    snippet text,
    read_at timestamptz NOT NULL DEFAULT now(),
    PRIMARY KEY (query, url)
    );
  4. 4. Write the function

    A Neon Function has a public URL, so the handler checks a caller token of your own first. Then it sends the question to the Search API and upserts each page from data.sources:

    functions/websearch.ts
    import { Hono } from "hono";
    import { attachDatabasePool } from "@neon/functions";
    import { Pool } from "pg";
    const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 5 });
    attachDatabasePool(pool);
    const app = new Hono();
    app.post("/search", async (c) => {
    if (c.req.header("Authorization") !== `Bearer ${process.env.CALLER_TOKEN}`) {
    return c.json({ error: "Unauthorized" }, 401);
    }
    const { q } = await c.req.json<{ q: string }>();
    const res = await fetch(
    `https://api.specstream.ai/v1/web-search?q=${encodeURIComponent(q)}`,
    { headers: { Authorization: `Bearer ${process.env.SPECSTREAM_API_KEY}` } },
    );
    if (!res.ok) return c.json({ error: `SpecStream answered ${res.status}` }, 502);
    const { data } = await res.json();
    for (const source of data.sources) {
    await pool.query(
    `INSERT INTO web_sources (query, url, title, snippet) VALUES ($1, $2, $3, $4)
    ON CONFLICT (query, url) DO UPDATE
    SET title = EXCLUDED.title, snippet = EXCLUDED.snippet, read_at = now()`,
    [q, source.url, source.title, source.snippet],
    );
    }
    return c.json({ answer: data.answer?.value ?? null, saved: data.sources.length });
    });
    export default app;
  5. 5. Declare it in neon.ts

    The slug websearch becomes part of the function's URL and can't be renamed later. Both keys are declared under env with required: true, so a deploy without them fails instead of shipping a function that can't search:

    neon.ts
    import { defineConfig } from "@neon/config/v1";
    export default defineConfig({
    functions: {
    websearch: {
    name: "SpecStream web search",
    source: "./functions/websearch.ts",
    env: {
    SPECSTREAM_API_KEY: { value: process.env.SPECSTREAM_API_KEY, required: true },
    CALLER_TOKEN: { value: process.env.CALLER_TOKEN, required: true },
    },
    },
    },
    });
  6. 6. Deploy and call it

    Add both keys to the .env.local file that neon link wrote, and keep that file out of Git. Make the caller token with openssl rand -hex 32. Deploy, read the function's URL from neon functions get websearch, and send a question:

    Terminal
    echo "SPECSTREAM_API_KEY=YOUR_SPECSTREAM_API_KEY" >> .env.local
    echo "CALLER_TOKEN=$(openssl rand -hex 32)" >> .env.local
    neon deploy --env .env.local
    neon functions get websearch
    set -a; . ./.env.local; set +a
    FUNCTION_URL="<the URL that neon functions get printed>"
    curl -X POST "${FUNCTION_URL%/}/search" \
    -H "Authorization: Bearer $CALLER_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{"q": "Postgres 18 release date"}'

Read the results

The pages are rows in your own database, so any client that reads Neon reads them: SELECT url, title, snippet, read_at FROM web_sources WHERE query = 'Postgres 18 release date' ORDER BY read_at DESC;. Asking the same question again refreshes the rows it already holds.

To read the full text of a page, call POST https://api.specstream.ai/v1/extract from the same function with {"urls": ["<page>"], "format": "markdown"} and store data.results[0].text in a column of its own.

Develop locally

neon dev --env .env.local serves the function at a localhost address with the branch's DATABASE_URL, so you can test the handler before you deploy. To run the search on a schedule instead of over HTTP, see Neon's Function Triggers (opens in a new tab).

Errors

  • 401: the key is missing or wrong. Send it as Authorization: Bearer <key>.
  • 403: the key has no Authorize paid work, or the call would go over the key's spending limit.
  • 402: the balance is empty, or the call would go over the account's monthly spending limit.
  • 503 with Retry-After: wait that many seconds and send it again.

Cost

Each call of the function is one search at $5 per 1,000 searches of up to 10 results, or $1 per 1,000 if you add &speed=fast to the address. A page read is $0.80 per 1,000 pages. They come out of your SpecStream balance up to the key's spending limit, with no quote and no approval. Neon bills the function's compute as usual. Balance, quotes, and billing has the details.

Remove

Delete the function with the Neon CLI or the Console, and drop web_sources if you no longer need the rows. To stop the key working everywhere, revoke it under API keys in Agents & API. Results already delivered stay in your workspace. Questions go to [email protected].

More

SpecStream for Neon covers what it does in Neon on one page. Every other tool has its guide in Connect your tools. You see the most a request can cost before it starts, and the matches are paid from your balance. Billing has the details. For help, email [email protected].