Hosted Stores: Block-Aware Key/Value Lookups for Substreams, Fully Managed
Substreams is great at turning raw blockchain data into structured output. Where it gets awkward is when a module needs to look something up mid-stream: a price that lives off chain, a token's metadata, a mapping from account to owner that another package already computed. Your options have been to recompute that data in every pipeline that needs it, bake it into a static file, or stand up and operate your own key/value service that stays correct across chain reorgs. All three are work you would rather not own.
Hosted Stores removes that work. A Hosted Store is a managed, organization-scoped, block-aware key/value store that StreamingFast runs for you on The Graph Market. You put data in, either by pushing it over gRPC from your own service (Remote Feed) or by pointing a Substreams package at it (Substreams Feed), and any Substreams module (or any gRPC client) reads it back at a specific block height, in sync with the stream. No service to operate, no reorg handling to write.
Hosted Stores joins Hosted Sinks under Hosted Services: the managed side of Substreams on The Graph Market, where StreamingFast operates the infrastructure and you just configure it.
What is a Hosted Store?
A Hosted Store is a key → value map with blockchain semantics:
- Keys are raw bytes: a string, an address, a hash, or any composite key you choose.
- Values are protobuf messages, tagged with their type (
google.protobuf.Any). You define the message; it is not a StreamingFast-defined shape. - Reads from a Substreams module happen at the block currently being processed, automatically, so lookups stay consistent with the stream even during backfills.
Your Hosted Store is consumed by your Substreams module through its deployment ID and version, <deployment-id>@v1.0.0 (the version is always v1.0.0 for now), and only your organization can access it.
What is a Hosted Store best used for?
A Hosted Store is useful whenever a Substreams module needs reference data that is not in the block it is processing. Typical jobs:
- resolving token metadata like symbol and decimals
- mapping an address to its owner or protocol
- checking an allowlist or denylist
- attaching an off-chain price to an on-chain event
- reading a value that another package has already computed
Because the store versions its data by block, a lookup during a historical backfill returns what was true at that block rather than only the latest value. A single store can serve many pipelines at once, so it is a good home for any lookup that more than one team would otherwise build and maintain on its own.
Two ways to fill a store
Hosted Stores come with two feed modes, both available from The Graph Market's new Hosted Store wizard.
Substreams feed: a package populates the store from chain data
Pick Substreams feed when the lookup is derived from the chain. A Substreams package reads blocks and writes key/value entries; the store handles reorgs automatically because Substreams is the source.
Example: a shared on-chain lookup store. ERC-20 token metadata (symbol, decimals, name) is expensive to resolve and every pricing, portfolio, or accounting pipeline needs it. Instead of each team's package doing its own eth_call dance, one package populates a Hosted Store with address → metadata, and every downstream Substreams module reads it as a foundational store. Same pattern for a DEX pool registry, an account-to-owner map on Solana, or a contract-to-protocol labeling set. Compute it once, reuse it everywhere, and new pipelines start from a warm lookup instead of a cold backfill.
Remote feed: you push data in over gRPC
Pick Remote feed when the data is off chain or application-owned: your service is the source of truth and Substreams just needs to look it up.
You write batches of entries with the Feed.Set gRPC call, delete with Feed.Delete, and once the data consumers need is in place, flip the store live with Feed.SetReady. Data pushed this way is treated as final. The store also supports setting multiple values for a key at different block heights, so you can handle advanced use cases.
Example: an off-chain price feed. Say your protocol settles against an index price computed off chain. Your pricing service pushes the latest value per asset into a Remote Feed store keyed by asset ID. A Substreams module that scores liquidations or marks positions declares the store as an input and calls store.get(...) per block; every block it processes sees the price your service had published by then, historical backfills included. No oracle contract to deploy, no price table to recompute in each pipeline.
Other good fits: allowlists and denylists, KYC / identity attestations, off-chain order metadata, feature flags — any application-owned reference data you want joined against chain events.
How do I read a Hosted Store from a Substreams module?
Declare it as a module input in substreams.yaml:
modules:
- name: map_enriched_events
kind: map
inputs:
- source: sf.ethereum.type.v2.Block
- foundational-store: <deployment-id>@v1.0.0
output:
type: proto:my.project.v1.EnrichedEvents
The runtime injects a FoundationalStore into your handler; call store.get(&[key]) and you get back one result per key, read at the block being processed:
#[substreams::handlers::map]
fn map_enriched_events(block: Block, store: FoundationalStore) -> Result<EnrichedEvents, Error> {
let response = store.get(&[address.as_slice()]);
// response.entries[0].code == ResponseCode::Found -> use entry value
// ...
}
You can also query the store directly from any gRPC client with Store.Get (exact keys) or Store.GetFirst (first key ≥ the one you asked for, handy for range scans).
One behavior to know: a Substreams module that queries a store which has not reached the current block hangs at that block (it does not error or time out) until the store catches up (or, for a Remote Feed store, until you call Feed.SetReady). This is deliberate: the runtime treats "not reached yet" as "try again shortly," which is what keeps reads consistent during backfills. Direct gRPC queries return immediately with block_reached: false instead. If a pipeline looks stuck right after you added a store input, the store almost certainly isn't ready yet.
When do I use a Hosted Store instead of a Hosted Sink?
They move data in opposite directions.
- A Hosted Sink takes Substreams output and writes it out to your Postgres or ClickHouse database, so you can query it with SQL.
- A Hosted Store holds data you look up from inside Substreams while a pipeline runs, and serves it back over gRPC too.
If the end consumer is a dashboard, an app backend, or a data pipeline that speaks SQL, you want a Hosted Sink. If a Substreams module needs reference data to do its job (enrich an event, filter by an allowlist, resolve an address), you want a Hosted Store.
For those deciding whether to build or buy
Every lookup store you run yourself is a small platform team's worth of standing work: a gRPC service to keep up, reorg logic to get right, a backend to back up, block-versioning to maintain, credentials to rotate. It is undifferentiated infrastructure. It does not make your product better, it just has to not break.
Hosted Services is the offer to hand that class of work to StreamingFast. Hosted Sinks means you do not operate a sink process to get chain data into your database. Hosted Stores means you do not operate a key/value service to do lookups inside your pipelines. Both run on The Graph Market's infrastructure, both are a form and a deploy button, and both let your engineers spend their cycles on the thing you are actually trying to ship.
Getting started
- Sign in to The Graph Market and go to the Hosted Services tab.
- Create a Hosted Store, choose Remote feed or Substreams feed, give it a name and a value type URL.
- Copy the deployment ID; it is both the gRPC hostname and the store reference in a manifest.
- Fill it: push entries with
Feed.Set+Feed.SetReady, or deploy the producer package. - Read it from a Substreams module via
foundational-store: <deployment-id>@<version>, or directly over gRPC.
Hosted Stores is in beta. It works end to end today; we are still smoothing edges and would like your feedback: join the Discord and tell us what you hit.