How to index blockchain data for a web3 application

A web3 application gets onchain data into its backend in three ways. It can poll an RPC endpoint, query a subgraph, or stream decoded data into its own database with a pipeline. Goldsky runs all three: Edge RPC, Subgraphs, and Turbo Pipelines. Each path puts the work for reorgs, backfills, and provider failures in a different place. Polling keeps that work in the team's code. A subgraph moves it to the host and gives the app a GraphQL API. A Turbo pipeline moves it to Goldsky's ingestion layer and delivers the data to a database the team already runs.

Goldsky measured all three paths on one Uniswap pool on Ethereum. Each one made a new event queryable about 5 seconds after its block. For history, a Turbo pipeline loaded 1,000,000 blocks into a database in 7.9 minutes, and a subgraph took 108.6 minutes for the same range.

The three approaches compared

Goldsky runs all three in production: Edge RPC, Subgraphs, and Turbo Pipelines. The table shows where each one puts the work for reorgs, backfills, and rate limits.

Chain counts come from Goldsky's chain list on November 10, 2026, which covers 150 chains and 271 networks across all products.

Measured: freshness, backfill, and read latency

Goldsky ran all three paths on November 10, 2026, against one contract: the Uniswap v3 USDC/WETH 0.05% pool on Ethereum, Swap event. Each path read the same 1,000,000 blocks. All three returned the same 812,171 events.

Freshness does not separate the three paths. Over 61 minutes and 223 blocks, each path made a new Swap event queryable about 4 to 5 seconds after the block timestamp, at the median. An Ethereum block comes every 12 seconds, so all three are current before the next block arrives.

Backfill does separate them. A Turbo pipeline decoded the 1,000,000 blocks and wrote them to a ClickHouse table in 7.9 minutes. A Goldsky subgraph took 108.6 minutes for the same range. This pool averages about 0.8 swaps per block, and Goldsky's product chooser states that subgraph latency grows as event frequency climbs.

An Edge poller reads the same raw logs quickly with eth_getLogs, but that covers only the read. The team still writes the decode step, the database writes, and the reorg handling.

All three answer a read in under a second. One subgraph query returns at most 1,000 rows. The last 1,000 blocks held 1,303 swaps, so a backend that reads a full window from a subgraph must page through the results. Edge and the ClickHouse table returned all 1,303 in one call.

The client ran in London. The ClickHouse table is in AWS us-west-2. Edge and the subgraph run in several regions. The post's example below uses a Postgres sink, and the benchmark used ClickHouse. Both are keyed sinks on the same Turbo delivery layer.

Poll an RPC endpoint with Goldsky Edge

This fits a backend that reads current state, such as a balance, an owner, or a price, at a known block. It fits less well when the backend must replay history or react to every event.

The team builds a client that calls eth_getLogs or eth_getBlockByNumber on a schedule, decodes the response, and writes it to its own store. A backfill means a loop over block ranges, sized to the endpoint's limit on blocks per request. The team also builds retries, a cursor for the last processed block, and a key that drops duplicate writes after a retry.

Edge RPC (opens in a new tab) takes over the provider-side part of that work. It serves requests from at least 8 regions and caches common reads. It sends latency-sensitive calls to several nodes at the same time, and it fails over to a healthy node inside the same request. Edge costs $5 per million requests, with every method at the same price. RPC providers differ in error rate and latency on the same chain and method. Goldsky's live RPC provider comparison measures that difference every hour from Edge traffic.

Edge does not detect reorgs in the team's own store, and it does not write the backfill loop. A team that polls still owns those two parts. That cost is small for a low-volume read of current state. It grows once the backend needs full history, or must never miss an event.

Query a Goldsky subgraph

This fits a dApp frontend that queries structured onchain data through an API, where the data maps to entities and events.

The team writes a schema and mapping handlers. It can also build a subgraph from a contract address and ABI with no mapping code, through Goldsky's no-code wizard (opens in a new tab). Goldsky runs the indexer and the GraphQL server.

Goldsky's product chooser (opens in a new tab) states that subgraph latency grows as event frequency climbs, and that a multi-chain subgraph often needs a reindex from its start block. A subgraph needs the least infrastructure for the team to run. In exchange, the data takes the shape of the subgraph's entity model, and it lives behind a GraphQL endpoint rather than in the team's database.

Stream data into your own database with a Turbo pipeline

This fits an application that joins chain data with its own business data. It also fits a team that needs a non-EVM chain such as Solana. It also fits a team that wants the data in the database it already runs.

The team writes a pipeline definition in YAML with a source, a transform, and a sink. Goldsky runs ingestion and delivery. The team runs the sink database and the queries against it.

Turbo gives at-least-once delivery. It detects a reorg at ingestion and walks back the affected deletes and updates. A keyed sink such as Postgres, ClickHouse, or MySQL then converges to the canonical chain with no manual step, per Turbo's delivery guarantees (opens in a new tab). A sink with no primary key, such as Kafka or a webhook, receives the same correction records, and the consumer applies them. A pipeline reads from Goldsky's curated datasets, not from an RPC node, so it has no rate limit on the read side.

A Turbo pipeline moves reorg handling and backfills from the team's code into Goldsky's ingestion layer. The team keeps its own database and query layer.

Index with Goldsky

A Turbo pipeline that decodes one contract's events into typed Postgres rows has three parts. Goldsky runs the ingestion step that reads and decodes the chain data. The team writes these three parts and operates the destination database.

• A raw log source, filtered to the contract address.
• A SQL transform that decodes the event and casts its fields.
• A Postgres sink, keyed on a stable ID.

This example from Goldsky's pipeline cookbook (opens in a new tab) decodes OrderFilled events from an exchange contract. The same three-part shape applies to any EVM event. Replace the address and the event signature with the contract the team watches:

name: exchange-trades
resource_size: s

sources:
  raw_logs:
    type: dataset
    dataset_name: ethereum.raw_logs
    version: 1.0.0
    start_at: earliest
    # Fast scan: pre-filter at the source so the backfill skips irrelevant blocks.
    # Replace the address with your contract.
    filter: >-
      address = '0x1111111111111111111111111111111111111111'
      AND block_number >= 21000000

transforms:
  # Step 1: decode raw logs with an inline ABI
  decoded_events:
    type: sql
    primary_key: id
    sql: |
      SELECT
        _gs_log_decode(
          '[{"anonymous":false,"inputs":[{"indexed":true,"name":"maker","type":"address"},{"indexed":true,"name":"taker","type":"address"},{"indexed":false,"name":"amount","type":"uint256"},{"indexed":false,"name":"price","type":"uint256"}],"name":"OrderFilled","type":"event"}]',
          topics,
          data
        ) AS decoded,
        id, block_number, transaction_hash, address, block_timestamp
      FROM raw_logs

  # Step 2: extract typed columns from the decoded event
  trades:
    type: sql
    primary_key: id
    sql: |
      SELECT
        id,
        block_number,
        block_timestamp,
        transaction_hash,
        address AS contract_address,
        decoded.event_params[1] AS maker,
        decoded.event_params[2] AS taker,
        (CAST(decoded.event_params[3] AS DOUBLE) / 1e18) AS amount,
        (CAST(decoded.event_params[4] AS DOUBLE) / 1e6) AS price_usdc
      FROM decoded_events
      WHERE decoded.event_signature = 'OrderFilled'

sinks:
  database:
    type: postgres
    from: trades
    schema: public
    table: trades
    secret_name: MY_POSTGRES_SECRET
    primary_key: id

Validate the pipeline, then deploy it:

goldsky turbo validate pipeline.yaml
goldsky turbo apply pipeline.yaml -i

How to index Solana data into Postgres with Goldsky Turbo takes this pattern to a live pipeline. It adds the Postgres secret setup and the backfill flags. Mirror is Goldsky's retired pipeline engine. A team that still runs it can read Turbo vs Mirror before it builds a new pipeline.

FAQ

Does polling an RPC endpoint handle chain reorgs automatically?
No. The endpoint returns the chain as it is at the time of the call. Code that detects a changed block and corrects data already written from it is the team's to write.

Can a subgraph and a Turbo pipeline run on the same data?
Yes. A subgraph entity source streams a deployed subgraph's entities into a pipeline. The team gets the GraphQL API and its own copy of the data in a database, at the same time.

How far back can a Turbo pipeline backfill?
Full history from genesis, on EVM chains and on Solana.

Which approach needs the least ongoing operations?
A Goldsky subgraph, since Goldsky runs indexing and the query layer. A Turbo pipeline adds a database to operate. Polling Edge RPC needs the poller, the reorg logic, and the backfill loop, all written and run by the team.

How do I start?
For a pipeline, install the Turbo CLI extension and follow the Turbo Pipelines quickstart (opens in a new tab). For an RPC endpoint, create an Edge endpoint in the Goldsky app.