How Koi Finance Unlocked 10x Faster Indexing and Querying with Goldsky

Koi Finance is a zkRollup-based DEX, farming platform, and bond platform built on Ethereum and zkSync. Koi features both a concentrated and standard pool AMM DEX, along with limit orders, farming, and bonds. As of Nov 2023, Koi has processed over 450M in volume with ~18M in TVL.

After experiencing significant performance issues with their existing subgraph infrastructure, Koi migrated to Goldsky in under 48 hours, immediately seeing a 10x increase in subgraph indexing speed and a 50x decrease in query response times.

Slow Subgraphs, Long Response Times

As the second-largest DEX on zkSync Era, Koi relies on subgraphs to power their mission-critical operations. With a total number of subgraph entities hitting 23 million and counting, the team began noticing critical scalability problems within their existing subgraph setup.

Certain queries powering Koi's frontend dashboards were taking over 10 seconds to return a response, leading to an incredibly slow and frustrating experience for end-users. Existing subgraph endpoints became overwhelmed with even moderate traffic, causing downtime and failed responses when data availability was needed the most.

Koi's engineering team knew their subgraphs could be optimized. But with every new deployment triggering a 6-7 day reindexing period, cycling through iterations would mean months of downtime for their 10k+ daily active users.

Goldsky Moment #1: Quickstart

On the hunt for a better solution, Koi explored a number of different subgraph providers before ultimately landing on Goldsky.

At the time, Goldsky didn't even support subgraph indexing on zkSync. But within 45 minutes of hearing from Koi, the Goldsky team spun up multiple RPC endpoints, integrated them into our load-balancing system, and deployed fully tested, end-to-end support for both zksync-era and zksync-era-testnet networks.

With just a single CLI command from Goldsky's subgraph migration workflow, Koi was able to get up and running on Goldsky in minutes.

Goldsky Moment #2: Subgraph Expertise

Koi began seeing subgraph performance improvements out of the box, but Goldsky's internal systems (benchmarking against thousands of deployments) triggered a manual review for additional optimizations.

Proactive Outreach from Goldsky's Support Team

Hey, noticed the subgraph indexing performance is pretty slow. We dug into the RPC layer and found that the overall responses were not as fast as our other supported networks, but also not completely out of bounds. Each RPC request takes around 100-200ms, mostly around the 100ms range.

We added and onboarded additional zksync providers to see if we could get better performance, but found that the performance actually decreased. At that point we dug into your subgraph logs — we found that one handler in particular, handleSync, was taking the majority of the time. On average, one handleSync takes 500ms to complete. The cost is likely related to the amount of eth_calls it has to make.

I'm wondering if you could share the subgraph source code so we can take a look — we might be able to find ways to increase efficiency.

After learning more about the subgraph in question, we identified key optimizations to recommend.

Subgraph Debugging and Improvement

Hey, after some digging, we found that handleSync does indeed make a bunch of eth_calls, in the findEthPerToken function. I'm thinking values can be pulled by using relational methods here.

We could do something like token.pairBase.load() and token.pairQuote.load(), concat them together, then iterate through the results whitelist instead.

The subgraph updates were quickly implemented, leading to an immediate improvement in critical performance metrics.

Outcome from Subgraph Rewrite

Nice! It's going around 10x faster. We're working on getting more high-quality nodes up to make that even faster.

A combination of Goldsky's infrastructure and support expertise meant Koi could now iterate on subgraphs in hours instead of weeks, enabling the team to quickly get to their desired end-state subgraph setup.

Goldsky Moment #3: Subgraph Database Customization

With the revamped subgraph in place, queries that would usually take 10+ seconds to process using The Graph (20+ seconds or complete failure at peak load) resolved in under 5 seconds using Goldsky.

Sample Query

{
    tokens(first: 100, orderBy: tradeVolumeUSD, orderDirection:desc){
      id
      symbol
      name
      decimals
      derivedETH
      totalLiquidity
      tradeVolumeUSD
    }
}

While the performance improvements were substantial, Goldsky was confident we could do even better.

One of the key benefits of Goldsky versus other subgraph hosts is the ability to enable optimizations at the underlying database-level, allowing users to define custom indexes based on specific query patterns. In Koi's case, we identified the right indexes to apply to the database, as well as advanced optimizations around nested relations. These improvements lead to the following:

Goldsky Team Quote

This query should feel a lot faster now!

Before: 73,828 ms (cold query)

After: 891 ms (cold query)

Supercharged Subgraphs, Powerful Results

With unrivaled indexing and querying performance, combined with responsive support and data expertise, Koi had Goldsky subgraphs live in production within 48 hours.

Matt Tawil, Lead Developer

Our experience on Goldsky has been better every step of the way: the product, the responsiveness, the expertise, the performance — wish we'd known about them earlier, it would have saved us a lot of headache.

Since then, Goldsky has been serving 50K+ requests a day to 10K+ daily active visitors without a single failed or dropped request. With a supercharged backend, Koi's team can get back to shipping high-value user experiences while pushing the frontier of DeFi on zkSync Era.

Realtime Data Streaming with Goldsky

Goldsky's mission is to unleash the full potential of Web3 data.

We're solving complex data engineering challenges so that builders can focus on what matters most — building transformational applications and delivering value to users.

Sign up for full access to Goldsky's realtime data streaming platform today, and be sure to give us a follow on Twitter for the latest news and announcements.