Back to Common Questions

Common Questions / Pipelines and databases

How do I look up reference data inside a Substreams module?

Updated

Use a Hosted Store. Declare it as an input to your module, and call store.get for the keys you need. Each lookup returns the value as of the block currently being processed, so a historical backfill sees what was true at each block.

Before this, the options were recomputing the data in every pipeline, baking it into a static file, or running your own key/value service that stays correct across reorgs.

How do I declare it?

Add it as a module input in substreams.yaml:

modules:
  - name: map_enriched_events
    kind: map
    inputs:
      - source: sf.ethereum.type.v2.Block
      - foundational-store: <deployment-id>@v1.0.0
    output:
      type: proto:my.project.v1.EnrichedEvents

The store is referenced by its deployment ID and version (currently always v1.0.0). The runtime injects a FoundationalStore into your handler, and store.get(&[key]) returns one result per key. See the foundational store reference.

Which kind of data fits?

  • Chain-derived, filled by a Substreams package: ERC-20 symbol, decimals and name; DEX pool registries; account-to-owner maps. Compute once, reuse across pipelines.
  • Application-owned, pushed over gRPC: an off-chain index price, allowlists and denylists, identity attestations, order metadata, feature flags.

Why not call the chain or an API from the module?

Substreams modules are deterministic and run in parallel across history. A lookup that depends on the latest external value would give different results on different runs; a block-versioned store gives the same answer every time.


Frequently asked questions

Can other teams read my store? No. It is scoped to your organization.

Can I read it without Substreams? Yes, directly over gRPC using the deployment ID as the hostname.


Get an API key at thegraph.market.

Related: What is a Hosted Store? · What is Substreams? · What is blockchain ETL?