Meltano
Load the live web data SpecStream finds into any target your Meltano project runs. The Singer tap tap-specstream reads your lists and records, with each value next to the page it was read from and when.
SpecStream for MeltanoData & warehouses
What you need
- A SpecStream account. Every new account starts with $10 of work and no card.
- A Meltano project on Python 3.10 or later.
- An API key from your SpecStream account, created in step 1.
- 1
1. Create an API key
Open Agents & API in SpecStream, choose API keys, and select Create API key. A read key with a $0 spending limit is all the tap needs.
Copy the key when SpecStream shows it. It won't show it again.
- 2
2. Add the tap
Add
tap-specstreamto your Meltano project from its definition file. The definition installs the `tap-specstream` package from PyPI (opens in a new tab).Terminal - 3
3. Set your key
Store the key as the tap's
api_keysetting. The tap marks it as sensitive. The optional settings are in the table below the steps.Terminal - 4
4. Add a loader and run
Any Singer target works. This example adds
target-duckdband runs the tap into it:Terminal
What arrives
The tap reads these streams:
| Stream | One row per | Fields |
|---|---|---|
lists | List | id, name, kind, imported, project_id, created_at, row_count, link |
records | Record, a child of lists | list_id, list_created_at, record_id, entity_id, values, cells |
Fields
kind is find, snapshot, import, or combined, and link opens the list in SpecStream.
values maps each column's label to its value. cells keeps every value with its source_url, the page it was read from, and its observed_at, when it was read. A value SpecStream couldn't find is null.
Records about one company, place, person, or posting share an entity_id in every list, so they join across lists. The tap supports stream maps, so you can rename or drop fields before they reach the target.
Settings
The tap takes these settings. Set an optional one the same way as the key, for example meltano config tap-specstream set start_date 2026-01-01T00:00:00Z.
| Setting | Required | What it does |
|---|---|---|
api_key | Yes | Your SpecStream API key. |
start_date | No | Reads only lists created after this time, for example 2026-01-01T00:00:00Z. |
api_url | No | The SpecStream API. Defaults to https://api.specstream.ai. |
Later runs
The first run reads every list the key can open. After that, the lists stream is incremental on created_at, so a run reads only lists created since the previous one. Meltano's state remembers where the last run stopped.
A SpecStream schedule puts what it finds in a fresh list. Your next run reads that list, so run meltano run on the scheduler you already use.
Use the tap without Meltano
Any Singer target can read it. Install the tap with pip install tap-specstream, put your settings in a config.json file with an api_key entry, and pipe the output to the target, for example tap-specstream --config config.json | target-jsonl.
Cost
The tap never starts a request or spends from your SpecStream balance, so every run is free. Your balance pays for the requests that find the data, only for results that match. The way quotes and billing work is in balance, quotes, and billing.
Disconnect
Remove the extractor from your project with meltano remove extractor tap-specstream, and revoke the key under Agents & API to end access at once. Data already in your target stays there. For the rest of the tools SpecStream connects to, see Connect your tools. Email [email protected] for help.
More
SpecStream for Meltano covers what it does in Meltano on one page. Every other tool has its guide in Connect your tools. You see the most a request can cost before it starts, and only the matches come out of your balance. Billing has the details. For help, email [email protected].