What is Substreams?

Updated August 2026

Substreams is a parallel data transformation engine for blockchains. You write small modules in Rust that describe which onchain data you want and how to shape it, and Substreams executes them across a chain's full history in parallel — then keeps streaming as new blocks arrive.

It is a compute layer, not a database or an API. The output goes wherever you send it: a Postgres or ClickHouse table, a Kafka topic, a subgraph, or straight into your application.

How is Substreams different from a normal indexer?

Most indexers process a chain block by block. That is fine at the chain head and painful for history — backfilling years sequentially can take days or weeks.

Substreams splits historical processing into segments and runs them in parallel, then switches to live streaming when it catches up. Full history and real-time data come from the same engine and the same module code.

Modules are also composable: one that decodes a contract's events can feed another that aggregates them, and either can be published for someone else to reuse. Ready-made packages live in the registry.

Do I have to write Rust?

No — and this is the part that has changed most recently. There are three ways to get a Substreams package, and only one of them involves writing Rust by hand.

Reuse a package. The registry has ready-made packages for common cases. Point one at a sink and you are done.

Describe what you want to an AI assistant. StreamingFast publishes five open-source agent skills that give AI coding assistants real expertise in Substreams: substreams-dev (manifests, Rust modules, protobuf schemas), substreams-sql (Postgres and ClickHouse sink schemas), substreams-sink (consuming output in Go, JavaScript, Python, Rust), substreams-sink-deploy (running sinks in production), and substreams-testing. They work in Claude Code, Cursor, and VS Code.

With them installed, you describe the data you want in plain language and the assistant builds the package.

This has been measured, not just asserted. In a published evaluation, an AI agent was given plain-English prompts for 14 tasks across Ethereum and Solana — from simple block statistics to Uniswap V3 pricing, Raydium CLMM swaps, and a Postgres sink. Every project built and ran successfully, and 12 of the 14 matched a golden reference output byte for byte. Two tasks were completed without an ABI or IDL at all, with the agent deriving what it needed from Solidity and Rust source.

The evaluation also documents where this breaks down — vague prompts produce confident guesses rather than clarifying questions, and non-standard ERC-20 tokens can cause field-decode failures. Be specific about what you want.

Write it yourself. Rust compiled to WebAssembly, in two module types:

	What it does	Typical use
map modules	Take a block, return extracted or transformed data	Decode transfers from a contract
store modules	Accumulate state across blocks	Running balances, totals, aggregates

You chain these into a pipeline defined by a manifest.

Where can Substreams send data?

To a database (PostgreSQL, ClickHouse, MongoDB, BigQuery), a message queue (Kafka, PubSub), a subgraph, or directly into your own application over gRPC. These destinations are called sinks, and they are the reason Substreams tends to show up in pipeline architecture decisions rather than in "which API should I call" decisions.

Which chains does it support?

Ethereum, Solana, Base, Arbitrum, Optimism, Polygon, BNB Chain, Avalanche, Bitcoin, NEAR, and Injective to name some of them. See The Graph Market for a complete list.

Substreams runs on Firehose, which extracts the underlying data. Firehose's data model is unusually deep — call trees, state changes, and balance changes, not just logs — so modules can reach data log-based indexers cannot see.

Frequently asked questions

Do I need to know Rust to use Substreams? No. You can reuse a package from the registry, or install the open-source Substreams agent skills and have an AI coding assistant build a custom package from a plain-language description. Writing Rust by hand is now the third option, not the first.

Is Substreams open source? Yes. Substreams and Firehose are developed in the open by StreamingFast under the Apache 2.0 licence, and can be self-hosted.

How is Substreams different from a subgraph? A subgraph is a hosted indexing definition that produces a GraphQL API. Substreams is a transformation engine that can output to many destinations, including a subgraph. Substreams-powered subgraphs use Substreams for the extraction and transformation step.

Substreams is permissionless — get an API key at thegraph.market and start streaming. No personal information required.