How do blockchain indexers work?

Updated August 2026

An indexer reads blocks from a blockchain, decodes the raw binary data into meaningful records, and writes those records to a store you can query. It does this for the chain's entire history, then keeps going as new blocks arrive.

The interesting engineering is in three places: how it gets the data, how it processes history without taking weeks, and what it does when the chain reorganises.

How does an indexer get data out of the chain?

Two approaches, and the choice shapes everything downstream.

Polling via RPC. The indexer repeatedly asks a node for new blocks and their logs. Simple, works with any standard node, and slow — every block costs round trips, and reading history means millions of them.

Instrumented extraction. The node itself emits everything as it executes, once, and that output is stored. Nothing is re-queried. This is how Firehose works, and it is why it can capture execution detail — call trees, state changes, balance changes — that never appears in logs at all.

How is historical data processed?

This is where indexers differ most in practice.

Sequentially. Process block 1, then block 2, and so on. Simple and correct, but the time to sync is proportional to chain length. For a large chain this can mean days or weeks, and it means every schema change starts that clock again.

In parallel. Split history into segments and process them simultaneously, then merge. Requires that transformations be structured so segments do not depend on each other — which is why engines that work this way impose a module structure rather than letting you write arbitrary code.

Substreams takes the parallel approach, then switches to streaming when it reaches the chain head. The same module code serves history and live data.

What happens when the chain reorganises?

Recent blocks are not final. A block your indexer has already processed can be replaced, and anything you derived from it is now wrong.

Naive indexers ignore this and hold incorrect data. Better ones wait a fixed number of blocks before treating anything as final, trading latency for safety. The most robust approach is a cursor — a token identifying exactly where in the chain a consumer is, including fork context — so a stream can describe what was undone rather than silently skipping it, and a disconnected consumer can resume at precisely the right point.

What does the output look like?

Whatever you need it to be. A GraphQL API, rows in Postgres or ClickHouse, messages on a Kafka topic, or a stream your application consumes directly.

The destination is called a sink. The reason it matters is that the shape of your output determines what queries are cheap later — an indexer that only offers one output shape has made that decision for you.

Frequently asked questions

Do all indexers handle reorgs? No. Some ignore them, some wait for a fixed confirmation depth, and some track fork state explicitly with cursors. It's worth checking, because the failure is silent.

Can I index a chain without running a node? Yes. Hosted endpoints provide indexed data without you operating infrastructure.

Why do some indexers take days to sync? Because they process history sequentially. Engines that parallelise historical processing avoid this.

Get indexed data without operating a node — API key at thegraph.market, no personal information required.