Skip to main content

Goldsky

Speed up subgraph indexing and queries

Talk to our team

What is subgraph optimization?

Subgraph optimization means cutting the time a subgraph takes to sync, backfill and reindex from its start block, and the query latency of its GraphQL endpoint, by changing how the subgraph is built and where it runs.

Teams whose backfills take days, whose schema changes wait on a long reindexing time or whose pages load slowly on heavy queries usually find the slowdown in the design: contract calls in every handler, growing arrays and stored history nobody queries.

You can speed up subgraph indexing and queries two ways: redeploy the subgraph on Goldsky Subgraphs and ship a leaner version of it, or serve the heaviest queries from your own database fed by Turbo Pipelines.

Who needs subgraph optimization?

DeFi protocols, analytics dashboards, wallets and games whose subgraphs take days to sync or reindex, or whose pages wait on slow GraphQL queries.

How a faster subgraph sync works

The diagram follows a subgraph from a sync that takes too long to a leaner version indexed on Goldsky and your app reading from it. Your setup may differ.

Your business, step 1: Sync takes too longDays to backfill or reindex
Goldsky, step 2: SubgraphsRedeploy on GoldskySame mappings on a rewritten RPC layer
Your business, steps 3–4: Make the subgraph leanerLess work per event, less storage
Goldsky, step 5: SubgraphsPrefetch contract callsDeclared calls answered from memory
Goldsky, step 6: SubgraphsReindex beside the old oneRe-sync reuses cached RPC results
Your business, step 8: Retire the old versionDelete what you no longer query
Goldsky, step 7: SubgraphsServe the same APIOne stable endpoint through a tag
  • Goldsky

Ways to speed up subgraph indexing and queries

Redeploying keeps the GraphQL API and speeds up sync and queries through Goldsky’s RPC layer and a leaner design; streaming moves heavy reads to a database you run and tune, so their query latency depends on your own database.

Usual setupOption 1: Redeploy and optimize
Best for
Apps that keep their GraphQL API and need faster syncs
Products
Subgraphs
What you run
Nothing; Goldsky runs indexing, storage and the API
What you write
A schema and mapping handlers, or a no-code or low-code config for simple contracts
Where your data lives
Hosted by Goldsky
How results reach you
You query; changes are pushed
Option 2: Serve from your database
Best for
Heavy aggregations and long histories
Products
Turbo
What you run
The destination: your database, queue or bucket, or a Postgres database Goldsky hosts
What you write
Pipeline YAML and SQL; TypeScript transforms when needed
Where your data lives
In your own database or systems
How results reach you
Pushed to you

Option 1: Speed up subgraph indexing by redeploying a leaner subgraph

You redeploy the subgraph on Goldsky, ship a leaner version beside it that does less work per event and stores less history, and move your app to it once it has synced. The GraphQL API your app queries stays the same.

When to redeploy and optimize a subgraph

Your app’s queries work and you want to keep the subgraph, but syncs and reindexes take too long or the indexer keeps falling behind.

What you’ll need

  • Goldsky Subgraphs
  • The subgraph’s source code, manifest and schema
  • Your app’s slowest queries, to test against
  • Access to the config where your app sets its subgraph URL

  1. Find where the sync spends its time

    A subgraph is slow for a reason, and the fix depends on whether it’s the work each event does, what the schema stores or the node the indexer reads from.

    Watch indexing progress across block ranges and look for handlers that call contracts, rewrite growing arrays or load entities they don’t change, and for queries that scan long histories. Speeding up the wrong part leaves a multi-day sync as it was.

  2. Redeploy on a faster RPC layer

    Handled by Goldsky Subgraphs

    Every contract call and log read in a sync goes through an RPC node, so a slow or failing node slows every block.

    The indexer needs fast, redundant access to the chain. An indexer that reads from one slow node stays slow however lean the subgraph is.

    Goldsky Subgraphs runs the same WASM mappings on a rewritten RPC layer, an autoscaling query layer and optimized storage, and you can deploy from the existing IPFS hash with --from-ipfs-hash or from source with --path.

    What Goldsky adds to subgraphs (opens in a new tab)

  3. Cut the work each event does

    Handler cost multiplies by event count, so a small waste in a busy handler becomes hours of sync time.

    Read values from the event instead of calling the contract, fetch fixed values such as a token’s decimals once and store them, and set the start block to the contract’s deployment block. Keep running totals in a plain field rather than reading a collection back in the handler.

  4. Shrink what the schema stores

    Storage the index keeps for every entity slows both indexing and queries, even when nothing reads it.

    Mark write-once entities immutable, store hashes and addresses as bytes, derive lists from the child entity instead of storing growing arrays, let the database compute rolling totals and prune history you never query. Most of these change the schema, so they ship as a new version.

  5. Declare the contract calls you can’t avoid

    Handled by Goldsky Subgraphs

    A contract call during indexing is a round trip to a node, the most expensive thing a handler can do.

    Calls computed only from event parameters can be listed in the manifest so the indexer fetches them before handlers run. A call that depends on state the mapping computed can’t be declared, and one whose arguments don’t match the handler’s falls back to a normal call.

    Goldsky Subgraphs runs declared eth_calls for each block in parallel before the handlers and caches the results in memory, so the handler’s contract call becomes a memory lookup.

    Declare eth_calls in the manifest (opens in a new tab)

  6. Reindex the lean version beside the old one

    Handled by Goldsky Subgraphs

    Schema changes mean a full reindex, and your app needs the old version answering until the new one catches up.

    Deploy the new version under a new name, leave the original running and compare real queries at the same block once the new one reaches the chain tip. Switching early shows users missing data.

    Goldsky permanently caches the RPC calls a subgraph makes while indexing, so a re-sync of the same or a similar subgraph reuses the cached eth_call and log results and runs faster than the first sync.

    Rebuild leaner and re-sync (opens in a new tab)

  7. Serve queries behind a stable endpoint

    Handled by Goldsky Subgraphs

    Your app needs to reach the new version without a release of its own.

    The endpoint your app calls should stay the same while the version behind it changes. Changing URLs in the app for every subgraph release adds a deploy to every fix.

    A Goldsky tag such as prod gives your app one GraphQL endpoint, and goldsky subgraph tag create moves it to the new version with no change on the app’s side.

    Move a tag to a new version (opens in a new tab)

  8. Retire the slow version

    An old version keeps indexing and storing data until someone deletes it.

    Once the app reads the new version, delete the old deployment. Every version left deployed keeps its worker and its stored entities, and is billed for both.

What to know about optimizing a subgraph

  • Most of the speedup comes from changes to the schema and mappings, which your team makes; the indexer runs standard subgraph code.
  • The biggest gains, such as immutable entities and Bytes ids, change the schema, so the new version reindexes from scratch rather than grafting.
  • Pruning history speeds up queries but removes time-travel queries below the pruned range.
  • Each deployed version is billed on its own until you delete the old one.

Option 2: Speed up subgraph queries by serving them from your own database

A pipeline backfills the same contracts’ events into a database you run, and your app sends its heaviest queries, such as large aggregations and long histories, to your own API on that database. The subgraph can keep serving the light queries, or be retired.

When to serve heavy subgraph queries from your own database

Your slowest queries scan long histories or aggregate across many entities, and you want their query latency set by a database you can index and scale yourself.

What you’ll need

  • Goldsky Turbo
  • The contract addresses, ABIs and start blocks the subgraph indexes
  • A database you run, such as ClickHouse for analytics or Postgres
  • An API layer for the queries you move

  1. Pick the queries that are too slow

    Moving every query is a rewrite; moving the slow ones is a fix.

    Rank your app’s GraphQL queries by latency and by how often pages call them, and note the contracts and events each one reads. Aggregations over long histories are usually the ones a general-purpose index serves slowest.

  2. Backfill the contracts’ history

    Handled by Goldsky Turbo

    Your database has to hold every past event the subgraph held before your app can read from it.

    The backfill starts at each contract’s start block and covers every event up to now. Scanning every block of the chain for a handful of contracts is slow, and a gap shows up later as missing records in the app.

    With a source filter on the contract addresses and start_at: earliest, Turbo Pipelines uses fast scan to skip blocks that hold none of your events during the backfill, then keeps streaming at the chain tip.

    Speed up backfills with fast scan (opens in a new tab)

  3. Decode the events

    Handled by Goldsky Turbo

    Raw logs are encoded, so each event has to be turned into named fields before your app can query it.

    Each contract’s events are decoded with its ABI into the columns your queries need. A wrong ABI decodes into nonsense without an error, so check a few rows against a block explorer.

    A Turbo Pipelines SQL transform decodes raw logs on the fly with _gs_log_decode and the contract’s ABI, so one pass can decode every event a contract emits and fan them out to a table per event.

    Decode contract events with an ABI (opens in a new tab)

  4. Stream events into tables built for the query

    Handled by Goldsky Turbo

    A table shaped and indexed for the query is what makes it fast.

    Choose the database and the table layout for how the query reads, such as a column store ordered by time for analytics. New rows have to keep arriving and rows from blocks a reorg replaced have to be corrected.

    The Turbo Pipelines ClickHouse sink creates a ReplacingMergeTree table ordered by your primary_key and marks rows a reorg removed as deleted, and Postgres and MySQL sinks upsert on the same key.

    Write to ClickHouse (opens in a new tab)

  5. Serve the heavy queries from your own API

    Your app needs an endpoint on your database for the queries you moved.

    Your team writes the queries against your tables, adds the indexes they need and points the slow pages at the new API. Compare results with the subgraph at the same block before switching.

  6. Keep or retire the subgraph

    Once the heavy queries have moved, the subgraph only has to serve what is left.

    Keep it for the light queries it already serves well, or move the rest and delete it. Two sources for the same screen without a rule for which wins leaves users seeing different numbers.

What to know about serving subgraph queries from your own database

  • Query latency depends on your database, its indexes and its size, which your team tunes.
  • A filtered backfill skips blocks that hold none of your contracts’ events, so history loads faster than a full scan of the chain.
  • The queries you move are rewritten against your own API; the rest can stay on the subgraph.
  • Tables with a primary key take reorg corrections as updates and deletes.

What you control

Your team
  • Find where the sync spends its time
  • Cut the work each event does
  • Shrink what the schema stores
  • Retire the slow version

Security and deployment

  • SOC 2 Type IIIndependently audited security controls.
  • Dedicated infrastructureSingle-tenant options for production workloads.
  • Data residencyCloud, on-premises or co-located.

More use cases that stream chain data to your systems

Use cases built the same way

Speed up subgraph indexing and queries with Goldsky

We’ll map it to your systems and show you what Goldsky runs, so your team builds the product, not the pipelines underneath it.

Talk to our team

© Endless Sky Inc. All rights reserved.

PrivacyTermsSecurity

System status