SpecStream for dltComing soon

What you get

• A records__cells tableEvery value becomes a row in records__cells with its label, source_url, and observed_at, tied to its record in records by _dlt_parent_id.
• Reruns merge in placeThe lists and records resources merge on their primary keys, so running it again updates the same rows. Each later run fetches only lists created since the one before.
• Swap the destination lineChange destination in dlt.pipeline() to go from DuckDB on a laptop to BigQuery, or edit shape_record to decide what each row carries.

Set it up

1. 1Install dlt with pip install "dlt[duckdb]" and copy specstream_pipeline.py from SpecStream's examples.
2. 2Export a key from Agents & API as SOURCES__SPECSTREAM__API_KEY. The pipeline only reads, so cap the key at $0.
3. 3Run python specstream_pipeline.py. The first run loads every list your key can open into the specstream_data dataset.

The full setup guide has every step, with what to do if something doesn’t arrive.

• SupabaseLive web records upserted into your Postgres table
• AirbyteComing soonYour lists and their records, in any Airbyte destination
• FivetranComing soonLIST, RECORD, and RECORD_VALUE tables in your warehouse
• Meltanotap-specstream from PyPI, into any Singer target