From ae755c16910c51872ae297eff66995931e5553b1 Mon Sep 17 00:00:00 2001 From: Brandon Kramer Date: Mon, 16 Nov 2026 19:01:14 +0000 Subject: [PATCH 1/4] Sync Supported Networks with registry v0.8.14, move Base to the Network template, and add Guides & Best Practices Supported Networks (registry v0.8.8 -> v0.8.14) - Add an Elysium page (Substreams via The Graph Market) - Remove pages for networks dropped from the table (Blast, Starknet, Starknet Testnet, Robinhood Chain Testnet) and redirect their routes to /supported-networks/ - zkSync Era: add Pinax Substreams section and "two ways" intro - Substreams overview and quick start: drop Starknet, Vara, and MANTRA Base - Use the BSC/Polygon template: longer intro, "Find Existing Subgraphs on Base" cards, Graph Network publishing flow Subgraphs: Guides & Best Practices - Merge How-to Guides into best-practices/ under the nav title "Guides & Best Practices"; rename "Tooling" to "Testing & Tooling" - Add the Subgraph Guides & Best Practices overview as the first page - Reorder pages to follow the overview, rewrite pages in one voice, and merge the two grafting pages into one - Redirect guides/, cookbook/, and grafting-hotfix URLs to their final pages in nginx.conf, and fix redirects to removed pages Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HpT75uYTePhuL7KFioUPqS --- nginx.conf | 55 ++- website/route-lockfile.txt | 18 +- website/src/HomePage.tsx | 4 +- .../en/indexing/new-chain-integration.mdx | 2 +- .../src/pages/en/subgraphs/_meta-titles.json | 5 +- website/src/pages/en/subgraphs/_meta.js | 1 - .../en/subgraphs/best-practices/_meta.js | 18 +- .../best-practices/avoid-eth-calls.mdx | 65 ++- .../subgraphs/best-practices/derivedfrom.mdx | 49 +-- .../{guides => best-practices}/enums.mdx | 11 +- .../best-practices/grafting-hotfix.mdx | 181 -------- .../en/subgraphs/best-practices/grafting.mdx | 246 +++++++++++ .../immutable-entities-bytes-as-ids.mdx | 64 ++- .../en/subgraphs/best-practices/overview.mdx | 411 ++++++++++++++++++ .../en/subgraphs/best-practices/pruning.mdx | 35 +- .../secure-api-keys-nextjs.mdx | 30 +- .../subgraph-composition.mdx | 34 +- .../best-practices/subgraph-debug-forking.mdx | 100 +++++ .../subgraphs/best-practices/timeseries.mdx | 51 +-- .../transfer-to-the-graph.mdx | 21 +- .../developing/creating/advanced.mdx | 2 +- .../developing/creating/subgraph-manifest.mdx | 2 +- .../src/pages/en/subgraphs/guides/_meta.js | 8 - .../pages/en/subgraphs/guides/grafting.mdx | 202 --------- .../guides/subgraph-debug-forking.mdx | 101 ----- .../src/pages/en/subgraphs/tooling/skills.mdx | 4 +- website/src/pages/en/substreams/overview.mdx | 2 +- .../src/pages/en/substreams/quick-start.mdx | 2 - .../supportedNetworks/customContent/README.md | 2 +- .../supportedNetworks/customContent/base.mdx | 55 ++- .../customContent/blast-mainnet.mdx | 9 - .../customContent/elysium.mdx | 9 + .../supportedNetworks/customContent/index.ts | 12 +- .../customContent/robinhood-sepolia.mdx | 9 - .../customContent/starknet-mainnet.mdx | 30 -- .../customContent/starknet-testnet.mdx | 30 -- .../customContent/zksync-era.mdx | 10 +- website/src/supportedNetworks/slugs.ts | 1 - 38 files changed, 1037 insertions(+), 854 deletions(-) rename website/src/pages/en/subgraphs/{guides => best-practices}/enums.mdx (92%) delete mode 100644 website/src/pages/en/subgraphs/best-practices/grafting-hotfix.mdx create mode 100644 website/src/pages/en/subgraphs/best-practices/grafting.mdx create mode 100644 website/src/pages/en/subgraphs/best-practices/overview.mdx rename website/src/pages/en/subgraphs/{guides => best-practices}/secure-api-keys-nextjs.mdx (64%) rename website/src/pages/en/subgraphs/{guides => best-practices}/subgraph-composition.mdx (74%) create mode 100644 website/src/pages/en/subgraphs/best-practices/subgraph-debug-forking.mdx rename website/src/pages/en/subgraphs/{guides => best-practices}/transfer-to-the-graph.mdx (73%) delete mode 100644 website/src/pages/en/subgraphs/guides/_meta.js delete mode 100644 website/src/pages/en/subgraphs/guides/grafting.mdx delete mode 100644 website/src/pages/en/subgraphs/guides/subgraph-debug-forking.mdx delete mode 100644 website/src/supportedNetworks/customContent/blast-mainnet.mdx create mode 100644 website/src/supportedNetworks/customContent/elysium.mdx delete mode 100644 website/src/supportedNetworks/customContent/robinhood-sepolia.mdx delete mode 100644 website/src/supportedNetworks/customContent/starknet-mainnet.mdx delete mode 100644 website/src/supportedNetworks/customContent/starknet-testnet.mdx diff --git a/nginx.conf b/nginx.conf index bde3913a84ca..55811434791b 100644 --- a/nginx.conf +++ b/nginx.conf @@ -58,30 +58,29 @@ http { rewrite ^/docs/en/about/introduction/$ $scheme://$http_host/docs/en/about/ permanent; rewrite ^/docs/en/billing/$ $scheme://$http_host/docs/en/subgraphs/providers/subgraph-studio/introduction/ permanent; rewrite ^/docs/en/chain-integration-overview/$ $scheme://$http_host/docs/en/indexing/chain-integration-overview/ permanent; - rewrite ^/docs/en/cookbook/arweave/$ $scheme://$http_host/docs/en/subgraphs/cookbook/arweave/ permanent; + rewrite ^/docs/en/cookbook/arweave/$ $scheme://$http_host/docs/en/supported-networks/ permanent; rewrite ^/docs/en/cookbook/avoid-eth-calls/$ $scheme://$http_host/docs/en/subgraphs/best-practices/avoid-eth-calls/ permanent; rewrite ^/docs/en/subgraphs/cookbook/avoid-eth-calls/$ $scheme://$http_host/docs/en/subgraphs/best-practices/avoid-eth-calls/ permanent; rewrite ^/docs/en/cookbook/derivedfrom/$ $scheme://$http_host/docs/en/subgraphs/best-practices/derivedfrom/ permanent; rewrite ^/docs/en/subgraphs/cookbook/derivedfrom/$ $scheme://$http_host/docs/en/subgraphs/best-practices/derivedfrom/ permanent; - rewrite ^/docs/en/cookbook/enums/$ $scheme://$http_host/docs/en/subgraphs/cookbook/enums/ permanent; - rewrite ^/docs/en/cookbook/grafting-hotfix/$ $scheme://$http_host/docs/en/subgraphs/best-practices/grafting-hotfix/ permanent; - rewrite ^/docs/en/subgraphs/cookbook/grafting-hotfix/$ $scheme://$http_host/docs/en/subgraphs/best-practices/grafting-hotfix/ permanent; - rewrite ^/docs/en/cookbook/grafting/$ $scheme://$http_host/docs/en/subgraphs/cookbook/grafting/ permanent; + rewrite ^/docs/en/cookbook/enums/$ $scheme://$http_host/docs/en/subgraphs/best-practices/enums/ permanent; + rewrite ^/docs/en/cookbook/grafting-hotfix/$ $scheme://$http_host/docs/en/subgraphs/best-practices/grafting/ permanent; + rewrite ^/docs/en/subgraphs/cookbook/grafting-hotfix/$ $scheme://$http_host/docs/en/subgraphs/best-practices/grafting/ permanent; + rewrite ^/docs/en/cookbook/grafting/$ $scheme://$http_host/docs/en/subgraphs/best-practices/grafting/ permanent; rewrite ^/docs/en/cookbook/immutable-entities-bytes-as-ids/$ $scheme://$http_host/docs/en/subgraphs/best-practices/immutable-entities-bytes-as-ids/ permanent; rewrite ^/docs/en/subgraphs/cookbook/immutable-entities-bytes-as-ids/$ $scheme://$http_host/docs/en/subgraphs/best-practices/immutable-entities-bytes-as-ids/ permanent; rewrite ^/docs/en/cookbook/near/$ $scheme://$http_host/docs/en/supported-networks/ permanent; rewrite ^/docs/en/cookbook/pruning/$ $scheme://$http_host/docs/en/subgraphs/best-practices/pruning/ permanent; rewrite ^/docs/en/subgraphs/cookbook/pruning/$ $scheme://$http_host/docs/en/subgraphs/best-practices/pruning/ permanent; rewrite ^/docs/en/cookbook/quick-start/$ $scheme://$http_host/docs/en/subgraphs/quick-start/ permanent; - rewrite ^/docs/en/cookbook/subgraph-debug-forking/$ $scheme://$http_host/docs/en/subgraphs/cookbook/subgraph-debug-forking/ permanent; - rewrite ^/docs/en/cookbook/subgraph-uncrashable/$ $scheme://$http_host/docs/en/subgraphs/cookbook/subgraph-uncrashable/ permanent; - rewrite ^/docs/en/cookbook/substreams-powered-subgraphs/$ $scheme://$http_host/docs/en/substreams/sps/introduction/ permanent; - rewrite ^/docs/en/subgraphs/cookbook/substreams-powered-subgraphs/$ $scheme://$http_host/docs/en/substreams/sps/introduction/ permanent; + rewrite ^/docs/en/cookbook/subgraph-debug-forking/$ $scheme://$http_host/docs/en/subgraphs/best-practices/subgraph-debug-forking/ permanent; + rewrite ^/docs/en/cookbook/subgraph-uncrashable/$ $scheme://$http_host/docs/en/subgraphs/tooling/subgraph-uncrashable/ permanent; + rewrite ^/docs/en/cookbook/substreams-powered-subgraphs/$ $scheme://$http_host/docs/en/substreams/overview/ permanent; + rewrite ^/docs/en/subgraphs/cookbook/substreams-powered-subgraphs/$ $scheme://$http_host/docs/en/substreams/overview/ permanent; rewrite ^/docs/en/cookbook/timeseries/$ $scheme://$http_host/docs/en/subgraphs/best-practices/timeseries/ permanent; rewrite ^/docs/en/subgraphs/cookbook/timeseries/$ $scheme://$http_host/docs/en/subgraphs/best-practices/timeseries/ permanent; - rewrite ^/docs/en/cookbook/transfer-to-the-graph/$ $scheme://$http_host/docs/en/subgraphs/cookbook/transfer-to-the-graph/ permanent; + rewrite ^/docs/en/cookbook/transfer-to-the-graph/$ $scheme://$http_host/docs/en/subgraphs/best-practices/transfer-to-the-graph/ permanent; rewrite ^/docs/en/subgraphs/cookbook/near/$ $scheme://$http_host/docs/en/supported-networks/ permanent; - rewrite ^/docs/en/subgraphs/cookbook/(.*)$ $scheme://$http_host/docs/en/subgraphs/guides/$1 permanent; rewrite ^/docs/en/curating/$ $scheme://$http_host/docs/en/resources/roles/curating/ permanent; rewrite ^/docs/en/delegating/$ $scheme://$http_host/docs/en/resources/roles/delegating/delegating/ permanent; rewrite ^/docs/en/deploying/deploy-using-subgraph-studio/$ $scheme://$http_host/docs/en/subgraphs/developing/deploying-publishing/using-subgraph-studio/ permanent; @@ -141,16 +140,16 @@ http { rewrite ^/docs/en/studio/billing/$ $scheme://$http_host/docs/en/subgraphs/providers/subgraph-studio/introduction/ permanent; rewrite ^/docs/en/studio/deploy-subgraph-studio/$ $scheme://$http_host/docs/en/subgraphs/developing/deploying-publishing/using-subgraph-studio/ permanent; rewrite ^/docs/en/studio/managing-api-keys/$ $scheme://$http_host/docs/en/subgraphs/providers/subgraph-studio/managing-api-keys/ permanent; - rewrite ^/docs/en/studio/multisig/$ $scheme://$http_host/docs/en/subgraphs/cookbook/multisig/ permanent; + rewrite ^/docs/en/studio/multisig/$ $scheme://$http_host/docs/en/subgraphs/providers/subgraph-studio/studio-faq/ permanent; rewrite ^/docs/en/studio/studio-faq/$ $scheme://$http_host/docs/en/subgraphs/providers/subgraph-studio/studio-faq/ permanent; rewrite ^/docs/en/studio/subgraph-studio/$ $scheme://$http_host/docs/en/subgraphs/developing/deploying-publishing/using-subgraph-studio/ permanent; rewrite ^/docs/en/studio/transferring-subgraph-ownership/$ $scheme://$http_host/docs/en/subgraphs/developing/managing/transferring-a-subgraph/ permanent; rewrite ^/docs/en/subgraphs/$ $scheme://$http_host/docs/en/subgraphs/overview/ permanent; rewrite ^/docs/en/substreams/$ $scheme://$http_host/docs/en/substreams/quick-start/ permanent; rewrite ^/docs/en/substreams/developing/sinks/sinks/$ $scheme://$http_host/docs/en/substreams/developing/sinks/ permanent; - rewrite ^/docs/en/substreams/sps/$ $scheme://$http_host/docs/en/substreams/sps/introduction/ permanent; + rewrite ^/docs/en/substreams/sps/$ $scheme://$http_host/docs/en/substreams/overview/ permanent; rewrite ^/docs/en/supported-network-requirements/$ $scheme://$http_host/docs/en/indexing/supported-network-requirements/ permanent; - rewrite ^/docs/en/supported-networks/arweave/$ $scheme://$http_host/docs/en/subgraphs/cookbook/arweave/ permanent; + rewrite ^/docs/en/supported-networks/arweave/$ $scheme://$http_host/docs/en/supported-networks/ permanent; rewrite ^/docs/en/supported-networks/near/$ $scheme://$http_host/docs/en/supported-networks/ permanent; rewrite ^/docs/en/tap/$ $scheme://$http_host/docs/en/indexing/tap/ permanent; rewrite ^/docs/en/tokenomics/$ $scheme://$http_host/docs/en/resources/tokenomics/ permanent; @@ -266,7 +265,7 @@ http { rewrite ^/docs/en/subgraphs/guides/contract-analyzer/$ $scheme://$http_host/docs/en/subgraphs/tooling/contract-analyzer/ permanent; rewrite ^/docs/en/substreams/skills/$ $scheme://$http_host/docs/en/substreams/tooling/skills/ permanent; rewrite ^/docs/en/substreams/substreams-mcp/(.*)$ $scheme://$http_host/docs/en/substreams/tooling/skills/ permanent; - rewrite ^/docs/en/resources/migration-guides/migrate-from-alchemy/$ $scheme://$http_host/docs/en/subgraphs/guides/transfer-to-the-graph/ permanent; + rewrite ^/docs/en/resources/migration-guides/migrate-from-alchemy/$ $scheme://$http_host/docs/en/subgraphs/best-practices/transfer-to-the-graph/ permanent; rewrite ^/docs/en/subgraphs/querying/managing-api-keys/$ $scheme://$http_host/docs/en/subgraphs/providers/subgraph-studio/managing-api-keys/ permanent; rewrite ^/docs/en/subgraphs/developing/creating/unit-testing-framework/$ $scheme://$http_host/docs/en/subgraphs/tooling/unit-testing-framework/ permanent; # --- round8 migration-guide redirects --- @@ -284,7 +283,6 @@ http { # --- supported networks: renamed custom chain page slugs --- rewrite ^/docs/en/supported-networks/btc/$ $scheme://$http_host/docs/en/supported-networks/bitcoin/ permanent; - rewrite ^/docs/en/supported-networks/blast-mainnet/$ $scheme://$http_host/docs/en/supported-networks/blast/ permanent; rewrite ^/docs/en/supported-networks/mainnet-cl/$ $scheme://$http_host/docs/en/supported-networks/ethereum-beacon/ permanent; rewrite ^/docs/en/supported-networks/hoodi-cl/$ $scheme://$http_host/docs/en/supported-networks/ethereum-hoodi/ permanent; rewrite ^/docs/en/supported-networks/sepolia-cl/$ $scheme://$http_host/docs/en/supported-networks/ethereum-sepolia/ permanent; @@ -303,6 +301,31 @@ http { rewrite ^/docs/en/supported-networks/viction/$ $scheme://$http_host/docs/en/supported-networks/ permanent; rewrite ^/docs/en/supported-networks/zksync-era-sepolia/$ $scheme://$http_host/docs/en/supported-networks/ permanent; + # --- supported networks: networks dropped from the table in registry v0.8.14 --- + rewrite ^/docs/en/supported-networks/blast/$ $scheme://$http_host/docs/en/supported-networks/ permanent; + rewrite ^/docs/en/supported-networks/blast-mainnet/$ $scheme://$http_host/docs/en/supported-networks/ permanent; + rewrite ^/docs/en/supported-networks/robinhood-sepolia/$ $scheme://$http_host/docs/en/supported-networks/ permanent; + rewrite ^/docs/en/supported-networks/starknet-mainnet/$ $scheme://$http_host/docs/en/supported-networks/ permanent; + rewrite ^/docs/en/supported-networks/starknet-testnet/$ $scheme://$http_host/docs/en/supported-networks/ permanent; + + # --- subgraphs: How-to Guides merged into Best Practices & Guides (subgraphs/best-practices/) --- + # Specific rules must stay above the catch-alls at the end of this block. + rewrite ^/docs/en/subgraphs/best-practices/grafting-hotfix/$ $scheme://$http_host/docs/en/subgraphs/best-practices/grafting/ permanent; + rewrite ^/docs/en/subgraphs/best-practices/$ $scheme://$http_host/docs/en/subgraphs/best-practices/overview/ permanent; + rewrite ^/docs/en/subgraphs/guides/$ $scheme://$http_host/docs/en/subgraphs/best-practices/overview/ permanent; + rewrite ^/docs/en/subgraphs/cookbook/$ $scheme://$http_host/docs/en/subgraphs/best-practices/overview/ permanent; + rewrite ^/docs/en/cookbook/$ $scheme://$http_host/docs/en/subgraphs/best-practices/overview/ permanent; + rewrite ^/docs/en/subgraphs/cookbook/arweave/$ $scheme://$http_host/docs/en/supported-networks/ permanent; + rewrite ^/docs/en/subgraphs/cookbook/multisig/$ $scheme://$http_host/docs/en/subgraphs/providers/subgraph-studio/studio-faq/ permanent; + rewrite ^/docs/en/subgraphs/cookbook/(agent0|polymarket)/$ $scheme://$http_host/docs/en/subgraphs/existing-subgraphs/$1/ permanent; + rewrite ^/docs/en/subgraphs/cookbook/(x402-payments|subgraph-uncrashable|subgraph-linter|contract-analyzer)/$ $scheme://$http_host/docs/en/subgraphs/tooling/$1/ permanent; + rewrite ^/docs/en/subgraphs/guides/(enums|grafting|secure-api-keys-nextjs|subgraph-composition|subgraph-debug-forking|transfer-to-the-graph)/$ $scheme://$http_host/docs/en/subgraphs/best-practices/$1/ permanent; + rewrite ^/docs/en/subgraphs/cookbook/subgraph-composition-three-sources/$ $scheme://$http_host/docs/en/subgraphs/best-practices/subgraph-composition/ permanent; + rewrite ^/docs/en/cookbook/subgraph-composition-three-sources/$ $scheme://$http_host/docs/en/subgraphs/best-practices/subgraph-composition/ permanent; + rewrite ^/docs/en/cookbook/(secure-api-keys-nextjs|subgraph-composition)/$ $scheme://$http_host/docs/en/subgraphs/best-practices/$1/ permanent; + rewrite ^/docs/en/subgraphs/cookbook/(.*)$ $scheme://$http_host/docs/en/subgraphs/best-practices/$1 permanent; + rewrite ^/docs/en/subgraphs/guides/(.*)$ $scheme://$http_host/docs/en/subgraphs/best-practices/$1 permanent; + location / { try_files $uri $uri.html $uri/index.html =404; } diff --git a/website/route-lockfile.txt b/website/route-lockfile.txt index 8fb7afb6123e..ef7631def74a 100644 --- a/website/route-lockfile.txt +++ b/website/route-lockfile.txt @@ -23,10 +23,16 @@ /en/resources/tokenomics/ /en/subgraphs/best-practices/avoid-eth-calls/ /en/subgraphs/best-practices/derivedfrom/ -/en/subgraphs/best-practices/grafting-hotfix/ +/en/subgraphs/best-practices/enums/ +/en/subgraphs/best-practices/grafting/ /en/subgraphs/best-practices/immutable-entities-bytes-as-ids/ +/en/subgraphs/best-practices/overview/ /en/subgraphs/best-practices/pruning/ +/en/subgraphs/best-practices/secure-api-keys-nextjs/ +/en/subgraphs/best-practices/subgraph-composition/ +/en/subgraphs/best-practices/subgraph-debug-forking/ /en/subgraphs/best-practices/timeseries/ +/en/subgraphs/best-practices/transfer-to-the-graph/ /en/subgraphs/developing/creating/advanced/ /en/subgraphs/developing/creating/assemblyscript-mappings/ /en/subgraphs/developing/creating/graph-node-dev/ @@ -49,12 +55,6 @@ /en/subgraphs/existing-subgraphs/explorer/ /en/subgraphs/existing-subgraphs/polymarket/ /en/subgraphs/existing-subgraphs/standard-subgraphs/ -/en/subgraphs/guides/enums/ -/en/subgraphs/guides/grafting/ -/en/subgraphs/guides/secure-api-keys-nextjs/ -/en/subgraphs/guides/subgraph-composition/ -/en/subgraphs/guides/subgraph-debug-forking/ -/en/subgraphs/guides/transfer-to-the-graph/ /en/subgraphs/overview/ /en/subgraphs/providers/subgraph-studio/fair-use-policy/ /en/subgraphs/providers/subgraph-studio/introduction/ @@ -100,7 +100,6 @@ /en/supported-networks/avalanche/ /en/supported-networks/base-sepolia/ /en/supported-networks/base/ -/en/supported-networks/blast-mainnet/ /en/supported-networks/bnb-svm/ /en/supported-networks/boba-testnet/ /en/supported-networks/boba/ @@ -113,6 +112,7 @@ /en/supported-networks/chapel/ /en/supported-networks/chiliz-testnet/ /en/supported-networks/chiliz/ +/en/supported-networks/elysium/ /en/supported-networks/eos/ /en/supported-networks/etherlink-mainnet/ /en/supported-networks/etherlink-shadownet/ @@ -171,8 +171,6 @@ /en/supported-networks/sonic-testnet/ /en/supported-networks/sonic/ /en/supported-networks/stable/ -/en/supported-networks/starknet-mainnet/ -/en/supported-networks/starknet-testnet/ /en/supported-networks/stellar-testnet/ /en/supported-networks/stellar/ /en/supported-networks/tempo-moderato/ diff --git a/website/src/HomePage.tsx b/website/src/HomePage.tsx index 6e5a0c12ce4c..b07effb0aef0 100644 --- a/website/src/HomePage.tsx +++ b/website/src/HomePage.tsx @@ -239,7 +239,7 @@ export default function HomePage({ supportedNetworks }: { supportedNetworks: Sup slotAboveTitle={} /> } @@ -251,7 +251,7 @@ export default function HomePage({ supportedNetworks }: { supportedNetworks: Sup slotAboveTitle={} /> } diff --git a/website/src/pages/en/indexing/new-chain-integration.mdx b/website/src/pages/en/indexing/new-chain-integration.mdx index 07dbe2ebbb91..dbebc4e1d172 100644 --- a/website/src/pages/en/indexing/new-chain-integration.mdx +++ b/website/src/pages/en/indexing/new-chain-integration.mdx @@ -47,7 +47,7 @@ For EVM chains, there exists a deeper level of data that can be achieved through ## EVM considerations - Difference between JSON-RPC & Firehose -While the JSON-RPC and Firehose are both suitable for Subgraphs, a Firehose is always required for developers wanting to build with [Substreams](https://substreams.streamingfast.io). Supporting Substreams allows developers to build [Substreams-powered Subgraphs](/subgraphs/cookbook/substreams-powered-subgraphs/) for the new chain, and has the potential to improve the performance of your Subgraphs. Additionally, Firehose — as a drop-in replacement for the JSON-RPC extraction layer of `graph-node` — reduces by 90% the number of RPC calls required for general indexing. +While the JSON-RPC and Firehose are both suitable for Subgraphs, a Firehose is always required for developers wanting to build with [Substreams](https://substreams.streamingfast.io). Supporting Substreams allows developers to build [Substreams-powered Subgraphs](https://thegraph.com/blog/substreams-powered-subgraphs/) for the new chain, and has the potential to improve the performance of your Subgraphs. Additionally, Firehose — as a drop-in replacement for the JSON-RPC extraction layer of `graph-node` — reduces by 90% the number of RPC calls required for general indexing. - All those `getLogs` calls and roundtrips get replaced by a single stream arriving into the heart of `graph-node`; a single block model for all Subgraphs it processes. diff --git a/website/src/pages/en/subgraphs/_meta-titles.json b/website/src/pages/en/subgraphs/_meta-titles.json index c9cd8f4959cc..aa322858c6ea 100644 --- a/website/src/pages/en/subgraphs/_meta-titles.json +++ b/website/src/pages/en/subgraphs/_meta-titles.json @@ -3,7 +3,6 @@ "providers": "Providers", "querying": "Querying", "developing": "Developing", - "guides": "How-to Guides", - "best-practices": "Best Practices", - "tooling": "Tooling" + "best-practices": "Guides & Best Practices", + "tooling": "Testing & Tooling" } diff --git a/website/src/pages/en/subgraphs/_meta.js b/website/src/pages/en/subgraphs/_meta.js index cbad796477b8..5caa12472886 100644 --- a/website/src/pages/en/subgraphs/_meta.js +++ b/website/src/pages/en/subgraphs/_meta.js @@ -9,5 +9,4 @@ export default { querying: titles.querying ?? '', tooling: titles.tooling ?? '', 'best-practices': titles['best-practices'] ?? '', - guides: titles.guides ?? '', } diff --git a/website/src/pages/en/subgraphs/best-practices/_meta.js b/website/src/pages/en/subgraphs/best-practices/_meta.js index 361d05d987ea..76467bdb8f08 100644 --- a/website/src/pages/en/subgraphs/best-practices/_meta.js +++ b/website/src/pages/en/subgraphs/best-practices/_meta.js @@ -1,8 +1,14 @@ export default { - 'immutable-entities-bytes-as-ids': '', - derivedfrom: '', - pruning: '', - 'avoid-eth-calls': '', - timeseries: '', - 'grafting-hotfix': '', + overview: 'Overview', + 'avoid-eth-calls': 'Avoid eth_calls', + 'immutable-entities-bytes-as-ids': 'Immutable Entities and Bytes IDs', + derivedfrom: 'Arrays with @derivedFrom', + timeseries: 'Timeseries and Aggregations', + enums: 'Categorize Data with Enums', + pruning: 'Pruning with indexerHints', + 'secure-api-keys-nextjs': 'Secure API Keys with Next.js', + 'subgraph-debug-forking': 'Debug with Subgraph Forking', + grafting: 'Grafting', + 'subgraph-composition': 'Subgraph Composition', + 'transfer-to-the-graph': 'Transfer to The Graph', } diff --git a/website/src/pages/en/subgraphs/best-practices/avoid-eth-calls.mdx b/website/src/pages/en/subgraphs/best-practices/avoid-eth-calls.mdx index 004accd0b69d..a4e7fd8fbadf 100644 --- a/website/src/pages/en/subgraphs/best-practices/avoid-eth-calls.mdx +++ b/website/src/pages/en/subgraphs/best-practices/avoid-eth-calls.mdx @@ -1,25 +1,22 @@ --- -title: Subgraph Best Practice 4 - Improve Indexing Speed by Avoiding eth_calls -sidebarTitle: 'Avoiding eth_calls' +title: Improve Indexing Speed by Avoiding eth_calls --- -## TLDR +An `eth_call` is a call from a Subgraph handler to a blockchain node to read contract state. Each one is an RPC round trip that takes a significant amount of time, so `eth_calls` are the most common cause of slow syncs. Where you can, design smart contracts to emit all the data your Subgraph needs, so handlers never need to call the contract. -`eth_calls` are calls that can be made from a Subgraph to an Ethereum node. These calls take a significant amount of time to return data, slowing down indexing. If possible, design smart contracts to emit all the data you need so you don’t need to use `eth_calls`. +## Why `eth_calls` Slow Down Indexing -## Why Avoiding `eth_calls` Is a Best Practice +Subgraphs are optimized to index event data emitted from smart contracts. A Subgraph can also index the data coming from an `eth_call`, however, this can significantly slow down Subgraph indexing as `eth_calls` require making external calls to smart contracts. Their speed depends not on the Subgraph but on the connectivity and responsiveness of the node being queried. Minimizing or eliminating `eth_calls` significantly improves indexing speed. -Subgraphs are optimized to index event data emitted from smart contracts. A Subgraph can also index the data coming from an `eth_call`, however, this can significantly slow down Subgraph indexing as `eth_calls` require making external calls to smart contracts. The responsiveness of these calls relies not on the Subgraph but on the connectivity and responsiveness of the Ethereum node being queried. By minimizing or eliminating eth_calls in our Subgraphs, we can significantly improve our indexing speed. +### What an `eth_call` Looks Like -### What Does an eth_call Look Like? +`eth_calls` are often necessary when the data required for a Subgraph is not available through emitted events. For example, consider a scenario where a Subgraph needs to identify whether ERC20 tokens are part of a specific pool, but the contract only emits a basic `Transfer` event that does not contain that data: -`eth_calls` are often necessary when the data required for a Subgraph is not available through emitted events. For example, consider a scenario where a Subgraph needs to identify whether ERC20 tokens are part of a specific pool, but the contract only emits a basic `Transfer` event and does not emit an event that contains the data that we need: - -```yaml +```solidity event Transfer(address indexed from, address indexed to, uint256 value); ``` -Suppose the tokens' pool membership is determined by a state variable named `getPoolInfo`. In this case, we would need to use an `eth_call` to query this data: +Suppose the tokens' pool membership is determined by a state variable named `getPoolInfo`. In this case, the handler needs an `eth_call` to read it: ```typescript import { Address } from '@graphprotocol/graph-ts' @@ -44,13 +41,13 @@ export function handleTransfer(event: Transfer): void { } ``` -This is functional, however is not ideal as it slows down our Subgraph’s indexing. +This works, but every `Transfer` now waits on an RPC call, which slows indexing. ## How to Eliminate `eth_calls` Ideally, the smart contract should be updated to emit all necessary data within events. For instance, modifying the smart contract to include pool information in the event could eliminate the need for `eth_calls`: -``` +```solidity event TransferWithPool(address indexed from, address indexed to, uint256 value, bytes32 indexed poolInfo); ``` @@ -73,17 +70,24 @@ export function handleTransferWithPool(event: TransferWithPool): void { } ``` -This is much more performant as it has eliminated the need for `eth_calls`. +This is much faster because the handler no longer makes any `eth_calls`. + +## How to Reduce `eth_calls` You Cannot Remove -## How to Optimize `eth_calls` +If you cannot change the contract, cut calls in this order: -If modifying the smart contract is not possible and `eth_calls` are required, read “[Improve Subgraph Indexing Performance Easily: Reduce eth_calls](https://thegraph.com/blog/improve-subgraph-performance-reduce-eth-calls/)” by Simon Emanuel Schmid to learn various strategies on how to optimize `eth_calls`. +1. **Read from the event:** Check `event.params` before you add any call. +2. **Cache values that never change:** Fetch values such as `name`, `symbol`, and `decimals` once, when the entity is created, and reuse the stored value. +3. **Compute in the Subgraph:** Derive values such as `totalSupply` from mint and burn events instead of calling the contract. +4. **Declare the calls you keep** (see the next section). + +For more strategies, read [Improve Subgraph Indexing Performance Easily: Reduce eth_calls](https://thegraph.com/blog/improve-subgraph-performance-reduce-eth-calls/). ## Reducing the Runtime Overhead of `eth_calls` -For the `eth_calls` that can not be eliminated, the runtime overhead they introduce can be minimized by declaring them in the manifest. When `graph-node` processes a block it performs all declared `eth_calls` in parallel before handlers are run. Calls that are not declared are executed sequentially when handlers run. The runtime improvement comes from performing calls in parallel rather than sequentially - that helps reduce the total time spent in calls but does not eliminate it completely. +For the `eth_calls` that can not be eliminated, the runtime overhead they introduce can be minimized by declaring them in the manifest. When Graph Node processes a block, it runs all declared `eth_calls` in parallel before the handlers run. Calls that are not declared are executed sequentially when handlers run. The runtime improvement comes from performing calls in parallel rather than sequentially - that helps reduce the total time spent in calls but does not eliminate it completely. -Currently, `eth_calls` can only be declared for event handlers. In the manifest, write +You can declare `eth_calls` only for event handlers, and only with `specVersion` 1.2.0 or later. In the manifest, add a `calls` section to the handler: ```yaml event: TransferWithPool(address indexed, address indexed, uint256, bytes32 indexed) @@ -92,26 +96,11 @@ calls: ERC20.poolInfo: ERC20[event.address].getPoolInfo(event.params.to) ``` -The portion highlighted in yellow is the call declaration. The part before the colon is simply a text label that is only used for error messages. The part after the colon has the form `Contract[address].function(params)`. Permissible values for address and params are `event.address` and `event.params.`. - -The handler itself accesses the result of this `eth_call` exactly as in the previous section by binding to the contract and making the call. graph-node caches the results of declared `eth_calls` in memory and the call from the handler will retrieve the result from this in memory cache instead of making an actual RPC call. - -Note: Declared eth_calls can only be made in Subgraphs with specVersion >= 1.2.0. - -## Conclusion - -You can significantly improve indexing performance by minimizing or eliminating `eth_calls` in your Subgraphs. - -## Subgraph Best Practices 1-6 - -1. [Improve Query Speed with Subgraph Pruning](/subgraphs/best-practices/pruning/) - -2. [Improve Indexing and Query Responsiveness by Using @derivedFrom](/subgraphs/best-practices/derivedfrom/) - -3. [Improve Indexing and Query Performance by Using Immutable Entities and Bytes as IDs](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) +The `calls` entry is the call declaration. The part before the colon is a text label used only in error messages. The part after the colon has the form `Contract[address].function(params)`. Permissible values for address and params are `event.address` and `event.params.`. -4. [Improve Indexing Speed by Avoiding `eth_calls`](/subgraphs/best-practices/avoid-eth-calls/) +The handler itself accesses the result of this `eth_call` exactly as in the previous section by binding to the contract and making the call. Graph Node caches the results of declared `eth_calls` in memory, so the call in the handler reads the cached result instead of making an RPC call. -5. [Simplify and Optimize with Timeseries and Aggregations](/subgraphs/best-practices/timeseries/) +## Learn More -6. [Use Grafting for Quick Hotfix Deployment](/subgraphs/best-practices/grafting-hotfix/) +- [Declared eth_calls](/subgraphs/developing/creating/advanced/#declared-eth_call): the manifest reference. +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the full checklist and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/best-practices/derivedfrom.mdx b/website/src/pages/en/subgraphs/best-practices/derivedfrom.mdx index 7fdf08258327..f4bc3c2bb739 100644 --- a/website/src/pages/en/subgraphs/best-practices/derivedfrom.mdx +++ b/website/src/pages/en/subgraphs/best-practices/derivedfrom.mdx @@ -1,15 +1,12 @@ --- -title: Subgraph Best Practice 2 - Improve Indexing and Query Responsiveness By Using @derivedFrom -sidebarTitle: 'Arrays with @derivedFrom' +title: Use @derivedFrom Instead of Growing Arrays --- -## TLDR - -Arrays in your schema can really slow down a Subgraph's performance as they grow beyond thousands of entries. If possible, the `@derivedFrom` directive should be used when using arrays as it prevents large arrays from forming, simplifies handlers, and reduces the size of individual entities, improving indexing speed and query performance significantly. +Arrays in your schema slow a Subgraph down as they grow beyond thousands of entries, because Graph Node copies the whole array on every update. Use the `@derivedFrom` directive for one-to-many relationships instead. It prevents large arrays from forming, simplifies handlers, and keeps entities small, which speeds up both indexing and queries. ## How to Use the `@derivedFrom` Directive -You just need to add a `@derivedFrom` directive after your array in your schema. Like this: +Add a `@derivedFrom` directive after the array field in your schema: ```graphql comments: [Comment!]! @derivedFrom(field: "post") @@ -21,9 +18,7 @@ comments: [Comment!]! @derivedFrom(field: "post") An example of a dynamically growing array is a blogging platform where a “Post” can have many “Comments”. -Let’s start with our two entities, `Post` and `Comment` - -Without optimization, you could implement it like this with an array: +Start with two entities, `Post` and `Comment`. Without optimization, you might store the comments as an array on the post: ```graphql type Post @entity { @@ -39,9 +34,9 @@ type Comment @entity { } ``` -Arrays like these will effectively store extra Comments data on the Post side of the relationship. +This array stores the comment references on the `Post` side of the relationship, and the array grows with every new comment. -Here’s what an optimized version looks like using `@derivedFrom`: +Here is the optimized version with `@derivedFrom`: ```graphql type Post @entity { @@ -58,31 +53,15 @@ type Comment @entity { } ``` -Just by adding the `@derivedFrom` directive, this schema will only store the “Comments” on the “Comments” side of the relationship and not on the “Post” side of the relationship. Arrays are stored across individual rows, which allows them to expand significantly. This can lead to particularly large sizes if their growth is unbounded. - -This will not only make our Subgraph more efficient, but it will also unlock three features: - -1. We can query the `Post` and see all of its comments. -2. We can do a reverse lookup and query any `Comment` and see which post it comes from. - -3. We can use [Derived Field Loaders](/subgraphs/developing/creating/graph-ts/api/#looking-up-derived-entities) to unlock the ability to directly access and manipulate data from virtual relationships in our Subgraph mappings. - -## Conclusion - -Use the `@derivedFrom` directive in Subgraphs to effectively manage dynamically growing arrays, enhancing indexing efficiency and data retrieval. - -For a more detailed explanation of strategies to avoid large arrays, check out Kevin Jones' blog: [Best Practices in Subgraph Development: Avoiding Large Arrays](https://thegraph.com/blog/improve-subgraph-performance-avoiding-large-arrays/). - -## Subgraph Best Practices 1-6 - -1. [Improve Query Speed with Subgraph Pruning](/subgraphs/best-practices/pruning/) - -2. [Improve Indexing and Query Responsiveness by Using @derivedFrom](/subgraphs/best-practices/derivedfrom/) +With `@derivedFrom`, the schema stores the relationship only on the `Comment` side, as a single `post` field. The `comments` field on `Post` is computed at query time, so it never grows in storage. -3. [Improve Indexing and Query Performance by Using Immutable Entities and Bytes as IDs](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) +This makes your Subgraph more efficient and gives you three capabilities: -4. [Improve Indexing Speed by Avoiding `eth_calls`](/subgraphs/best-practices/avoid-eth-calls/) +1. Query a `Post` and see all of its comments. +2. Do a reverse lookup: query any `Comment` and see which post it belongs to. +3. Use [derived field loaders](/subgraphs/developing/creating/graph-ts/api/#looking-up-derived-entities) (`loadRelated()`) to read virtual relationships in your mappings. Derived fields are read-only in mappings. -5. [Simplify and Optimize with Timeseries and Aggregations](/subgraphs/best-practices/timeseries/) +## Learn More -6. [Use Grafting for Quick Hotfix Deployment](/subgraphs/best-practices/grafting-hotfix/) +- [Best Practices in Subgraph Development: Avoiding Large Arrays](https://thegraph.com/blog/improve-subgraph-performance-avoiding-large-arrays/): more strategies for avoiding large arrays. +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the full checklist and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/guides/enums.mdx b/website/src/pages/en/subgraphs/best-practices/enums.mdx similarity index 92% rename from website/src/pages/en/subgraphs/guides/enums.mdx rename to website/src/pages/en/subgraphs/best-practices/enums.mdx index 9f55ae07c54b..b898f71bca87 100644 --- a/website/src/pages/en/subgraphs/guides/enums.mdx +++ b/website/src/pages/en/subgraphs/best-practices/enums.mdx @@ -1,8 +1,8 @@ --- -title: Categorize NFT Marketplaces Using Enums +title: Categorize Data with Enums --- -Use Enums to make your code cleaner and less error-prone. Here's a full example of using Enums on NFT marketplaces. +Use enums for fields that take one of a fixed set of values. Enums make your schema clearer and reject invalid values and typos. This guide walks through a full example that categorizes NFT marketplace activity. ## What are Enums? @@ -26,7 +26,7 @@ enum TokenStatus { This means that when you use the `TokenStatus` type in your schema, you expect it to be exactly one of predefined values: `OriginalOwner`, `SecondOwner`, or `ThirdOwner`, ensuring consistency and validity. -To learn more about enums, check out [Creating a Subgraph](/developing/creating-a-subgraph/#enums) and [GraphQL documentation](https://graphql.org/learn/schema/#enumeration-types). +To learn more about enums, check out [The Graph QL Schema](/subgraphs/developing/creating/ql-schema/#enums) and [GraphQL documentation](https://graphql.org/learn/schema/#enumeration-types). ## Benefits of Using Enums @@ -269,6 +269,7 @@ Expected output includes the marketplaces that meet the criteria, each represent } ``` -## Additional Resources +## Learn More -For additional information, check out this guide's [repo](https://github.com/chidubemokeke/Subgraph-Tutorial-Enums). +- Check out this guide's [example repo](https://github.com/chidubemokeke/Subgraph-Tutorial-Enums). +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the full checklist and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/best-practices/grafting-hotfix.mdx b/website/src/pages/en/subgraphs/best-practices/grafting-hotfix.mdx deleted file mode 100644 index bb5f1fdbdb18..000000000000 --- a/website/src/pages/en/subgraphs/best-practices/grafting-hotfix.mdx +++ /dev/null @@ -1,181 +0,0 @@ ---- -title: Subgraph Best Practice 6 - Use Grafting for Quick Hotfix Deployment -sidebarTitle: 'Grafting and Hotfixing' ---- - -## TLDR - -Grafting is a powerful feature in Subgraph development that allows you to build and deploy new Subgraphs while reusing the indexed data from existing ones. - -### Overview - -This feature enables quick deployment of hotfixes for critical issues, eliminating the need to re-index the entire Subgraph from scratch. By preserving historical data, grafting minimizes downtime and ensures continuity in data services. - -## Benefits of Grafting for Hotfixes - -1. **Rapid Deployment** - - **Minimize Downtime**: When a Subgraph encounters a critical error and stops indexing, grafting enables you to deploy a fix immediately without waiting for re-indexing. - - **Immediate Recovery**: The new Subgraph continues from the last indexed block, ensuring that data services remain uninterrupted. - -2. **Data Preservation** - - **Reuse Historical Data**: Grafting copies the existing data from the base Subgraph, so you don’t lose valuable historical records. - - **Consistency**: Maintains data continuity, which is crucial for applications relying on consistent historical data. - -3. **Efficiency** - - **Save Time and Resources**: Avoids the computational overhead of re-indexing large datasets. - - **Focus on Fixes**: Allows developers to concentrate on resolving issues rather than managing data recovery. - -## Best Practices When Using Grafting for Hotfixes - -1. **Initial Deployment Without Grafting** - - **Start Clean**: Always deploy your initial Subgraph without grafting to ensure that it’s stable and functions as expected. - - **Test Thoroughly**: Validate the Subgraph’s performance to minimize the need for future hotfixes. - -2. **Implementing the Hotfix with Grafting** - - **Identify the Issue**: When a critical error occurs, determine the block number of the last successfully indexed event. - - **Create a New Subgraph**: Develop a new Subgraph that includes the hotfix. - - **Configure Grafting**: Use grafting to copy data up to the identified block number from the failed Subgraph. - - **Deploy Quickly**: Publish the grafted Subgraph to restore service as soon as possible. - -3. **Post-Hotfix Actions** - - **Monitor Performance**: Ensure the grafted Subgraph is indexing correctly and the hotfix resolves the issue. - - **Republish Without Grafting**: Once stable, deploy a new version of the Subgraph without grafting for long-term maintenance. - > Note: Relying on grafting indefinitely is not recommended as it can complicate future updates and maintenance. - - **Update References**: Redirect any services or applications to use the new, non-grafted Subgraph. - -4. **Important Considerations** - - **Careful Block Selection**: Choose the graft block number carefully to prevent data loss. - - **Tip**: Use the block number of the last correctly processed event. - - **Use Deployment ID**: Ensure you reference the Deployment ID of the base Subgraph, not the Subgraph ID. - - **Note**: The Deployment ID is the unique identifier for a specific Subgraph deployment. - - **Feature Declaration**: Remember to declare grafting in the Subgraph manifest under features. - -## Example: Deploying a Hotfix with Grafting - -Suppose you have a Subgraph tracking a smart contract that has stopped indexing due to a critical error. Here’s how you can use grafting to deploy a hotfix. - -1. **Failed Subgraph Manifest (subgraph.yaml)** - - ```yaml - specVersion: 1.3.0 - schema: - file: ./schema.graphql - dataSources: - - kind: ethereum/contract - name: OldSmartContract - network: sepolia - source: - address: '0xOldContractAddress' - abi: Lock - startBlock: 5000000 - mapping: - kind: ethereum/events - apiVersion: 0.0.9 - language: wasm/assemblyscript - entities: - - Withdrawal - abis: - - name: Lock - file: ./abis/OldLock.json - eventHandlers: - - event: Withdrawal(uint256,uint256) - handler: handleOldWithdrawal - file: ./src/old-lock.ts - ``` - -2. **New Grafted Subgraph Manifest (subgraph.yaml)** - ```yaml - specVersion: 1.3.0 - schema: - file: ./schema.graphql - dataSources: - - kind: ethereum/contract - name: NewSmartContract - network: sepolia - source: - address: '0xNewContractAddress' - abi: Lock - startBlock: 6000001 # Block after the last indexed block - mapping: - kind: ethereum/events - apiVersion: 0.0.9 - language: wasm/assemblyscript - entities: - - Withdrawal - abis: - - name: Lock - file: ./abis/Lock.json - eventHandlers: - - event: Withdrawal(uint256,uint256) - handler: handleWithdrawal - file: ./src/lock.ts - features: - - grafting - graft: - base: QmBaseDeploymentID # Deployment ID of the failed Subgraph - block: 6000000 # Last successfully indexed block - ``` - -**Explanation:** - -- **Data Source Update**: The new Subgraph points to 0xNewContractAddress, which may be a fixed version of the smart contract. -- **Start Block**: Set to one block after the last successfully indexed block to avoid reprocessing the error. -- **Grafting Configuration**: - - **base**: Deployment ID of the failed Subgraph. - - **block**: Block number where grafting should begin. - -3. **Deployment Steps** - - **Update the Code**: Implement the hotfix in your mapping scripts (e.g., handleWithdrawal). - - **Adjust the Manifest**: As shown above, update the `subgraph.yaml` with grafting configurations. - - **Deploy the Subgraph**: - - Authenticate with the Graph CLI. - - Deploy the new Subgraph using `graph deploy`. - -4. **Post-Deployment** - - **Verify Indexing**: Check that the Subgraph is indexing correctly from the graft point. - - **Monitor Data**: Ensure that new data is being captured and the hotfix is effective. - - **Plan for Republish**: Schedule the deployment of a non-grafted version for long-term stability. - -## Warnings and Cautions - -While grafting is a powerful tool for deploying hotfixes quickly, there are specific scenarios where it should be avoided to maintain data integrity and ensure optimal performance. - -- **Incompatible Schema Changes**: If your hotfix requires altering the type of existing fields or removing fields from your schema, grafting is not suitable. Grafting expects the new Subgraph’s schema to be compatible with the base Subgraph’s schema. Incompatible changes can lead to data inconsistencies and errors because the existing data won’t align with the new schema. -- **Significant Mapping Logic Overhauls**: When the hotfix involves substantial modifications to your mapping logic—such as changing how events are processed or altering handler functions—grafting may not function correctly. The new logic might not be compatible with the data processed under the old logic, leading to incorrect data or failed indexing. -- **Deployments to The Graph Network**: Grafting is not recommended for Subgraphs intended for The Graph’s decentralized network (mainnet). It can complicate indexing and may not be fully supported by all Indexers, potentially causing unexpected behavior or increased costs. For mainnet deployments, it’s safer to re-index the Subgraph from scratch to ensure full compatibility and reliability. - -### Risk Management - -- **Data Integrity**: Incorrect block numbers can lead to data loss or duplication. -- **Testing**: Always test grafting in a development environment before deploying to production. - -## Conclusion - -Grafting is an effective strategy for deploying hotfixes in Subgraph development, enabling you to: - -- **Quickly Recover** from critical errors without re-indexing. -- **Preserve Historical Data**, maintaining continuity for applications and users. -- **Ensure Service Availability** by minimizing downtime during critical fixes. - -However, it’s important to use grafting judiciously and follow best practices to mitigate risks. After stabilizing your Subgraph with the hotfix, plan to deploy a non-grafted version to ensure long-term maintainability. - -## Additional Resources - -- **[Grafting Documentation](/subgraphs/cookbook/grafting/)**: Replace a Contract and Keep its History With Grafting -- **[Understanding Deployment IDs](/subgraphs/querying/subgraph-id-vs-deployment-id/)**: Learn the difference between Deployment ID and Subgraph ID. - -By incorporating grafting into your Subgraph development workflow, you can enhance your ability to respond to issues swiftly, ensuring that your data services remain robust and reliable. - -## Subgraph Best Practices 1-6 - -1. [Improve Query Speed with Subgraph Pruning](/subgraphs/best-practices/pruning/) - -2. [Improve Indexing and Query Responsiveness by Using @derivedFrom](/subgraphs/best-practices/derivedfrom/) - -3. [Improve Indexing and Query Performance by Using Immutable Entities and Bytes as IDs](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) - -4. [Improve Indexing Speed by Avoiding `eth_calls`](/subgraphs/best-practices/avoid-eth-calls/) - -5. [Simplify and Optimize with Timeseries and Aggregations](/subgraphs/best-practices/timeseries/) - -6. [Use Grafting for Quick Hotfix Deployment](/subgraphs/best-practices/grafting-hotfix/) diff --git a/website/src/pages/en/subgraphs/best-practices/grafting.mdx b/website/src/pages/en/subgraphs/best-practices/grafting.mdx new file mode 100644 index 000000000000..59ff840736c4 --- /dev/null +++ b/website/src/pages/en/subgraphs/best-practices/grafting.mdx @@ -0,0 +1,246 @@ +--- +title: Graft a Subgraph to Ship Fixes and Replace Contracts +--- + +Grafting lets you deploy a new Subgraph that starts from an existing Subgraph's indexed data instead of from scratch. Use it in two situations: to ship a fix for a Subgraph that failed on a bug, and to move to a contract redeployed at a new address while keeping the old contract's history. Both skip a full re-index. This page covers how grafting works, when not to use it, and a walkthrough for each scenario. + +## What Grafting Is + +When you deploy a Subgraph whose manifest contains a `graft` block, Graph Node copies the data of the `base` Subgraph up to and including the given `block`, then continues indexing the new Subgraph from that block on. Copying is much faster than indexing, though the initial copy can still take several hours for a very large Subgraph. + +The base Subgraph must exist on the same Graph Node and must have indexed at least up to the graft block. Because of this, treat a grafted Subgraph as temporary: use it during development or an emergency, then replace it with an equivalent non-grafted version. + +## Set Up Grafting + +Add two items to the new Subgraph's manifest: + +```yaml +features: + - grafting +graft: + base: Qm... # Deployment ID of the base Subgraph + block: 6000000 # Last block to copy from the base Subgraph +``` + +- **`features`:** Declare `grafting` here. Grafting does not work without it. +- **`graft.base`:** The **Deployment ID** of the base Subgraph (the `Qm...` IPFS hash), not its Subgraph ID. See [Subgraph ID vs Deployment ID](/subgraphs/querying/subgraph-id-vs-deployment-id/). +- **`graft.block`:** The last block to copy. Choose it carefully: for a failed Subgraph, use the block of the last correctly processed event. A wrong block can lose or duplicate data. + +### Compatible Schema Changes + +The grafted Subgraph's schema does not have to match the base schema exactly, but it must be compatible with it. It must be a valid schema on its own, and it may differ from the base schema only in these ways: + +- It adds or removes entity types +- It removes attributes from entity types +- It adds nullable attributes to entity types +- It turns non-nullable attributes into nullable attributes +- It adds values to enums +- It adds or removes interfaces +- It changes for which entity types an interface is implemented + +Any other change, such as changing a field's type or making a nullable field non-nullable, is not graft-compatible. Schema optimizations such as switching to [immutable entities or `Bytes` IDs](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) also break compatibility. + +## When Not to Use Grafting + +- **Subgraphs on The Graph Network:** Grafting is not recommended for Subgraphs published to The Graph Network, and some Indexers may not support it. Publish your first version to the network without grafting. You cannot graft a Subgraph from The Graph Network back to Subgraph Studio. +- **Incompatible schema changes:** If the fix needs a schema change outside the list above, re-index instead. +- **Large mapping rewrites:** If the fix changes how events are processed across the board, the new logic may not be consistent with data indexed under the old logic, which leads to incorrect data. +- **Pruned blocks:** You cannot graft at a block that has been [pruned](/subgraphs/best-practices/pruning/). If you graft often, keep a pruning window that covers it. +- **Speeding up the rest of a sync:** Grafting skips the blocks before the graft point only. Later blocks index faster only if the new code does less work per event. + +Always test a graft in a development environment before you rely on it in production. + +## Scenario 1: Ship a Fix for a Failed Subgraph + +When a Subgraph stops indexing because of a bug, grafting lets you deploy the fix right away. The new version copies everything the failed version indexed, so data services stay available and no history is lost. + +1. **Find the graft block:** Find the block of the last event the failed Subgraph processed correctly. +2. **Fix the mapping:** Implement the fix in your mapping code, for example in `handleWithdrawal`. +3. **Add the graft:** Point `graft.base` at the failed deployment's Deployment ID, and `graft.block` at the block from step 1. +4. **Deploy:** Authenticate with the Graph CLI and run `graph deploy`. +5. **Verify:** Confirm that indexing resumes from the graft point and that the fix resolves the error. + +Here is the manifest for the fixed version. The data source is unchanged; only the mapping and the `graft` block are new: + +```yaml +specVersion: 1.3.0 +schema: + file: ./schema.graphql +dataSources: + - kind: ethereum/contract + name: SmartContract + network: sepolia + source: + address: '0xYourContractAddress' + abi: Lock + startBlock: 5000000 + mapping: + kind: ethereum/events + apiVersion: 0.0.9 + language: wasm/assemblyscript + entities: + - Withdrawal + abis: + - name: Lock + file: ./abis/Lock.json + eventHandlers: + - event: Withdrawal(uint256,uint256) + handler: handleWithdrawal # fixed handler + file: ./src/lock.ts +features: + - grafting +graft: + base: QmFailedDeploymentID # Deployment ID of the failed Subgraph + block: 6000000 # Last correctly indexed block +``` + +## Scenario 2: Replace a Contract and Keep Its History + +When you redeploy a contract at a new address, for example after an upgrade, graft a new Subgraph that tracks the new contract onto the Subgraph that tracked the old one. The new Subgraph serves the old contract's history and the new contract's events from one endpoint. + +This walkthrough replaces a contract with an identical one (a new address, the same code) on Sepolia. + +### Build the Base Subgraph + +To build and deploy the existing Subgraph used in this guide, use the example repo below. For the basics of building a Subgraph, see the [Quick Start](/subgraphs/quick-start/). + +- [Subgraph example repo](https://github.com/Shiyasmohd/grafting-tutorial) + +> Note: The contract used in the Subgraph was taken from the following [Hackathon Starterkit](https://github.com/schmidsi/hackathon-starterkit). + +### Base Subgraph Manifest + +The Subgraph manifest `subgraph.yaml` identifies the data sources for the Subgraph, the triggers of interest, and the functions that should be run in response to those triggers. This is the manifest for the base Subgraph: + +```yaml +specVersion: 1.3.0 +schema: + file: ./schema.graphql +dataSources: + - kind: ethereum + name: Lock + network: sepolia + source: + address: '0xb3aabe721794b85fe4e72134795c2f93b4eb7e63' + abi: Lock + startBlock: 5955690 + mapping: + kind: ethereum/events + apiVersion: 0.0.9 + language: wasm/assemblyscript + entities: + - Withdrawal + abis: + - name: Lock + file: ./abis/Lock.json + eventHandlers: + - event: Withdrawal(uint256,uint256) + handler: handleWithdrawal + file: ./src/lock.ts +``` + +- The `Lock` data source is the ABI and contract address you get when you compile and deploy the contract +- The network should correspond to an indexed network being queried. Because this example runs on Sepolia testnet, the network is `sepolia` +- The `mapping` section defines the triggers of interest and the functions that should be run in response to those triggers. In this case, the Subgraph listens for the `Withdrawal` event and calling the `handleWithdrawal` function when it is emitted. + +### Deploy the Base Subgraph + +1. Go to [Subgraph Studio](https://thegraph.com/studio/) and create a Subgraph on Sepolia testnet called `graft-example` +2. Follow the directions in the `AUTH & DEPLOY` section on your Subgraph page in the `graft-example` folder from the repo +3. Once finished, verify the Subgraph is indexing properly. If you run the following command in The Graph Playground + +```graphql +{ + withdrawals(first: 5) { + id + amount + when + } +} +``` + +It returns something like this: + +``` +{ + "data": { + "withdrawals": [ + { + "id": "0xe8323d21c4f104607b10b0fff9fc24b9612b9488795dea8196b2d5f980d3dc1d0a000000", + "amount": "0", + "when": "1716394824" + }, + { + "id": "0xea1cee35036f2cacb72f2a336be3e54ab911f5bebd58f23400ebb8ecc5cfc45203000000", + "amount": "0", + "when": "1716394848" + } + ] + } +} +``` + +Once the base Subgraph is indexing properly, you can graft onto it. + +### Deploy the Grafted Subgraph + +The graft replacement subgraph.yaml will have a new contract address. This could happen when you update your dapp, redeploy a contract, etc. + +1. Go to [Subgraph Studio](https://thegraph.com/studio/) and create a Subgraph on Sepolia testnet called `graft-replacement` +2. Create a new manifest. The `subgraph.yaml` for `graft-replacement` contains a different contract address and new information about how it should graft. These are the `block` of the [last event emitted](https://sepolia.etherscan.io/tx/0xea1cee35036f2cacb72f2a336be3e54ab911f5bebd58f23400ebb8ecc5cfc452) you care about by the old contract and the `base` of the old Subgraph. The `base` Subgraph ID is the `Deployment ID` of your original `graft-example` Subgraph. You can find this in Subgraph Studio. +3. Follow the directions in the `AUTH & DEPLOY` section on your Subgraph page in the `graft-replacement` folder from the repo +4. Once finished, verify the Subgraph is indexing properly. If you run the following command in The Graph Playground + +```graphql +{ + withdrawals(first: 5) { + id + amount + when + } +} +``` + +It should return the following: + +``` +{ + "data": { + "withdrawals": [ + { + "id": "0xe8323d21c4f104607b10b0fff9fc24b9612b9488795dea8196b2d5f980d3dc1d0a000000", + "amount": "0", + "when": "1716394824" + }, + { + "id": "0xea1cee35036f2cacb72f2a336be3e54ab911f5bebd58f23400ebb8ecc5cfc45203000000", + "amount": "0", + "when": "1716394848" + }, + { + "id": "0x2410475f76a44754bae66d293d14eac34f98ec03a3689cbbb56a716d20b209af06000000", + "amount": "0", + "when": "1716429732" + } + ] + } +} +``` + +You can see that the `graft-replacement` Subgraph is indexing from older `graft-example` data and newer data from the new contract address. The original contract emitted two `Withdrawal` events, [Event 1](https://sepolia.etherscan.io/tx/0xe8323d21c4f104607b10b0fff9fc24b9612b9488795dea8196b2d5f980d3dc1d) and [Event 2](https://sepolia.etherscan.io/tx/0xea1cee35036f2cacb72f2a336be3e54ab911f5bebd58f23400ebb8ecc5cfc452). The new contract emitted one `Withdrawal` after, [Event 3](https://sepolia.etherscan.io/tx/0x2410475f76a44754bae66d293d14eac34f98ec03a3689cbbb56a716d20b209af). The two previously indexed transactions (Event 1 and 2) and the new transaction (Event 3) were combined together in the `graft-replacement` Subgraph. + +The grafted Subgraph now serves the full history across both contracts. + +## After Grafting + +- **Republish without grafting:** Once the grafted version is stable, deploy a non-grafted version for long-term maintenance. Relying on a graft indefinitely complicates future updates. +- **Move your apps:** Point every app and service at the new, non-grafted version. +- **Keep the old version serving** until the new one has fully synced. + +## Learn More + +- [Grafting onto existing Subgraphs](/subgraphs/developing/creating/advanced/#grafting-onto-existing-subgraphs): the manifest reference. +- [Debug with Subgraph Forking](/subgraphs/best-practices/subgraph-debug-forking/): find and fix the bug locally before you graft. +- Grafting examples for popular contracts: [Curve](https://github.com/messari/subgraphs/blob/master/subgraphs/curve-finance/protocols/curve-finance/config/templates/curve.template.yaml), [ERC-721](https://github.com/messari/subgraphs/blob/master/subgraphs/erc721-metadata/subgraph.yaml), and [Uniswap](https://github.com/messari/subgraphs/blob/master/subgraphs/uniswap-v3-forks/protocols/uniswap-v3/config/templates/uniswapV3Template.yaml). +- [Data source templates](/subgraphs/developing/creating/subgraph-manifest/#data-source-templates): another way to handle contracts that change over time. +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the full checklist and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/best-practices/immutable-entities-bytes-as-ids.mdx b/website/src/pages/en/subgraphs/best-practices/immutable-entities-bytes-as-ids.mdx index 2e2b27a6cbb2..6c97913f8b85 100644 --- a/website/src/pages/en/subgraphs/best-practices/immutable-entities-bytes-as-ids.mdx +++ b/website/src/pages/en/subgraphs/best-practices/immutable-entities-bytes-as-ids.mdx @@ -1,15 +1,12 @@ --- -title: Subgraph Best Practice 3 - Improve Indexing and Query Performance by Using Immutable Entities and Bytes as IDs -sidebarTitle: 'Immutable Entities and Bytes as IDs' +title: Use Immutable Entities and Bytes as IDs --- -## TLDR - -Using Immutable Entities and Bytes for IDs in our `schema.graphql` file [significantly improves ](https://thegraph.com/blog/two-simple-subgraph-performance-improvements/) indexing speed and query performance. +Two schema changes [significantly improve](https://thegraph.com/blog/two-simple-subgraph-performance-improvements/) indexing speed, query performance, and storage: marking entities that never change as immutable, and using `Bytes` for IDs. ## Immutable Entities -To make an entity immutable, we simply add `(immutable: true)` to an entity. +To make an entity immutable, add `(immutable: true)` to it: ```graphql type Transfer @entity(immutable: true) { @@ -20,21 +17,21 @@ type Transfer @entity(immutable: true) { } ``` -By making the `Transfer` entity immutable, graph-node is able to process the entity more efficiently, improving indexing speeds and query responsiveness. +When the `Transfer` entity is immutable, Graph Node processes it more efficiently, which speeds up indexing and queries. -Immutable Entities structures will not change in the future. An ideal entity to become an Immutable Entity would be an entity that is directly logging onchain event data, such as a `Transfer` event being logged as a `Transfer` entity. +An immutable entity never changes after it is created. The best candidates are entities that log onchain event data directly, such as a `Transfer` event stored as a `Transfer` entity. -### Under the hood +### Under the Hood -Mutable entities have a 'block range' indicating their validity. Updating these entities requires the graph node to adjust the block range of previous versions, increasing database workload. Queries also need filtering to find only live entities. Immutable entities are faster because they are all live and since they won't change, no checks or updates are required while writing, and no filtering is required during queries. +Mutable entities have a 'block range' indicating their validity. Updating these entities requires Graph Node to adjust the block range of previous versions, increasing database workload. Queries also need filtering to find only live entities. Immutable entities are faster because they are all live and since they won't change, no checks or updates are required while writing, and no filtering is required during queries. -### When not to use Immutable Entities +### When Not to Use Immutable Entities -If you have a field like `status` that needs to be modified over time, then you should not make the entity immutable. Otherwise, you should use immutable entities whenever possible. +If an entity has a field that changes over time, such as `status`, do not make it immutable: a write to an immutable entity causes an indexing error. Otherwise, use immutable entities whenever possible. ## Bytes as IDs -Every entity requires an ID. In the previous example, we can see that the ID is already of the Bytes type. +Every entity requires an ID. In the previous example, the ID is already of type `Bytes`. ```graphql type Transfer @entity(immutable: true) { @@ -45,19 +42,19 @@ type Transfer @entity(immutable: true) { } ``` -While other types for IDs are possible, such as String and Int8, it is recommended to use the Bytes type for all IDs due to character strings taking twice as much space as Byte strings to store binary data, and comparisons of UTF-8 character strings must take the locale into account which is much more expensive than the bytewise comparison used to compare Byte strings. +Other ID types, such as `String` and `Int8`, are possible, but use `Bytes` for all IDs where you can. Character strings take twice as much space as byte strings to store binary data, and comparing UTF-8 strings must take the locale into account, which costs much more than the bytewise comparison used for `Bytes`. ### Reasons to Not Use Bytes as IDs -1. If entity IDs must be human-readable such as auto-incremented numerical IDs or readable strings, Bytes for IDs should not be used. -2. If integrating a Subgraph’s data with another data model that does not use Bytes as IDs, Bytes as IDs should not be used. -3. Indexing and querying performance improvements are not desired. +1. Entity IDs must be human-readable, such as auto-incremented numbers or readable strings. +2. The Subgraph's data must integrate with another data model that does not use `Bytes` IDs. +3. Indexing and query performance are not a concern for this entity. ### Concatenating With Bytes as IDs -It is a common practice in many Subgraphs to use string concatenation to combine two properties of an event into a single ID, such as using `event.transaction.hash.toHex() + "-" + event.logIndex.toString()`. However, as this returns a string, this significantly impedes Subgraph indexing and querying performance. +It is a common practice in many Subgraphs to use string concatenation to combine two properties of an event into a single ID, such as using `event.transaction.hash.toHex() + "-" + event.logIndex.toString()`. Because this returns a string, it significantly slows indexing and queries. -Instead, we should use the `concatI32()` method to concatenate event properties. This strategy results in a `Bytes` ID that is much more performant. +Instead, use the `concatI32()` method to concatenate event properties. The result is a `Bytes` ID that is much faster to store and compare. ```typescript export function handleTransfer(event: TransferEvent): void { @@ -76,7 +73,7 @@ export function handleTransfer(event: TransferEvent): void { ### Sorting With Bytes as IDs -Sorting using Bytes as IDs is not optimal as seen in this example query and response. +Sorting by a `Bytes` ID does not give numeric order, as this example query and response show. Query: @@ -120,9 +117,9 @@ Query response: } ``` -The IDs are returned as hex. +The IDs are returned and sorted as hex. -To improve sorting, we should create another field on the entity that is a BigInt. +To sort by a number, add a `BigInt` field to the entity: ```graphql type Transfer @entity { @@ -134,7 +131,7 @@ type Transfer @entity { } ``` -This will allow for sorting to be optimized sequentially. +Then order by that field: Query: @@ -170,22 +167,11 @@ Query Response: } ``` -## Conclusion - -Using both Immutable Entities and Bytes as IDs has been shown to markedly improve Subgraph efficiency. Specifically, tests have highlighted up to a 28% increase in query performance and up to a 48% acceleration in indexing speeds. - -Read more about using Immutable Entities and Bytes as IDs in this blog post by David Lutterkort, a Software Engineer at Edge & Node: [Two Simple Subgraph Performance Improvements](https://thegraph.com/blog/two-simple-subgraph-performance-improvements/). - -## Subgraph Best Practices 1-6 - -1. [Improve Query Speed with Subgraph Pruning](/subgraphs/best-practices/pruning/) - -2. [Improve Indexing and Query Responsiveness by Using @derivedFrom](/subgraphs/best-practices/derivedfrom/) - -3. [Improve Indexing and Query Performance by Using Immutable Entities and Bytes as IDs](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) +## Results -4. [Improve Indexing Speed by Avoiding `eth_calls`](/subgraphs/best-practices/avoid-eth-calls/) +In a controlled test, using immutable entities and `Bytes` IDs together cut time per block by 28%, sync time by 26%, and storage by 48%. See [Benchmarks and Evidence](/subgraphs/best-practices/overview/#benchmarks-and-evidence) for the full comparison. -5. [Simplify and Optimize with Timeseries and Aggregations](/subgraphs/best-practices/timeseries/) +## Learn More -6. [Use Grafting for Quick Hotfix Deployment](/subgraphs/best-practices/grafting-hotfix/) +- [Two Simple Subgraph Performance Improvements](https://thegraph.com/blog/two-simple-subgraph-performance-improvements/): the original benchmark by David Lutterkort. +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the full checklist and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/best-practices/overview.mdx b/website/src/pages/en/subgraphs/best-practices/overview.mdx new file mode 100644 index 000000000000..1ea4b6d54d52 --- /dev/null +++ b/website/src/pages/en/subgraphs/best-practices/overview.mdx @@ -0,0 +1,411 @@ +--- +title: Subgraph Guides & Best Practices +sidebarTitle: Overview +--- + +Most slow Subgraphs are slow because of their design or their RPC source, not their hardware. The fixes are usually small schema, manifest, or mapping changes. This section collects the guides and best practices for building Subgraphs that sync fast, answer queries fast, and stay healthy. This page is the map: it gives you a checklist, explains each practice, and links to the page in this section that covers it in depth. + +**Where to start:** Use the checklist below. For a Subgraph that already syncs slowly, go to [Speeding Up Sync Time and Iteration](#speeding-up-sync-time-and-iteration) first to find the bottleneck. + +## Pages in This Section + +The pages follow the order of this overview, from contract design to migration. + +| Page | Use it to | +| --- | --- | +| [Avoid eth_calls](/subgraphs/best-practices/avoid-eth-calls/) | Design contract events so handlers read data from events, and declare the calls you keep | +| [Immutable Entities and Bytes IDs](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) | Cut storage and speed up writes and queries with immutable entities and `Bytes` IDs | +| [Arrays with @derivedFrom](/subgraphs/best-practices/derivedfrom/) | Model one-to-many relationships without arrays that grow | +| [Timeseries and Aggregations](/subgraphs/best-practices/timeseries/) | Let the database compute hourly and daily metrics | +| [Categorize Data with Enums](/subgraphs/best-practices/enums/) | Restrict fields to a fixed set of valid values | +| [Pruning with indexerHints](/subgraphs/best-practices/pruning/) | Drop history you do not query to keep the database small | +| [Secure API Keys with Next.js](/subgraphs/best-practices/secure-api-keys-nextjs/) | Keep your API key out of frontend code | +| [Debug with Subgraph Forking](/subgraphs/best-practices/subgraph-debug-forking/) | Reproduce a failure at block X without syncing to block X | +| [Grafting](/subgraphs/best-practices/grafting/) | Ship a fix for a failed Subgraph, or move to a redeployed contract, without a full re-index | +| [Subgraph Composition](/subgraphs/best-practices/subgraph-composition/) | Build a Subgraph on top of up to five source Subgraphs | +| [Transfer to The Graph](/subgraphs/best-practices/transfer-to-the-graph/) | Move an existing Subgraph from another platform to The Graph Network | + +## Subgraph Performance Checklist + +Check each item before you publish. Items in the first group have the largest effect on sync time. + +**Sync speed** + +- `startBlock` is the contract's deployment block, not 0 +- Event handlers only; no block handlers without a filter, and call handlers only where events are missing +- No `eth_call` for data that is already in `event.params` +- Values that never change (`name`, `symbol`, `decimals`) are fetched once and stored on an entity +- Remaining `eth_calls` are declared in the manifest (`specVersion` 1.2.0 or higher) +- Handlers save only the entities they changed +- Only the events the schema needs have handlers; topic filters narrow the rest + +**Schema and storage** + +- Event-log entities use `@entity(immutable: true)` +- IDs use `Bytes`, built with `concatI32()`, not string concatenation +- One-to-many relationships use `@derivedFrom`, not arrays that grow +- Time-bucketed metrics use timeseries and aggregations (`specVersion` 1.1.0 or higher) +- `indexerHints.prune` is set (`auto`, or a block count if you need Time Travel Queries or grafting) +- Numeric fields that you sort by are `Int` or `BigInt`, not `String` + +**Reliability** + +- Contract calls that can revert use `try_` with a default value +- Subgraph Linter and Matchstick unit tests pass +- Grafting, if used for a hotfix, is followed by a non-grafted republish + +## Start with the Smart Contract + +The fastest Subgraph reads everything it needs from events. If you can still change the contract, [design its events for indexing](https://thegraph.com/blog/subgraph-devex-tips/). This removes the need for most `eth_calls`, which are [the most common cause of slow syncs](https://thegraph.com/blog/subgraph-optimization-tips/). + +- **Emit an event for every state change:** A Subgraph rebuilds contract state from events. Stake, unstake, claim, and configuration changes each need an event. +- **Put the full payload in the event:** Include the address, amount, and any context the Subgraph needs. An event that carries only an ID forces a contract call for every other value. +- **Balance detail against gas:** Each extra event field costs gas. Include only what the Subgraph needs, not every possible field. +- **Index the arguments you will filter on:** Indexed arguments enable topic filters in the manifest (see the next section). + +```solidity +// Forces eth_calls: the Subgraph must look up the user and amount +event Stake(uint256 indexed positionId); + +// Indexing-ready: all data is in the event +event Stake(uint256 indexed positionId, address indexed user, uint256 amount); +``` + +The official [Avoid eth_calls](/subgraphs/best-practices/avoid-eth-calls/) page shows the same pattern: replace a `Transfer` event plus a `getPoolInfo()` call with a `TransferWithPool` event that carries `poolInfo` directly. + +## Manifest: Start Blocks, Handlers, and Filters + +The [manifest](/subgraphs/developing/creating/subgraph-manifest/) decides how many blocks and triggers the Subgraph processes. Fewer blocks and fewer triggers mean a faster sync. + +| Setting | What to do | Why | Min `specVersion` | +| --- | --- | --- | --- | +| `startBlock` | Set it to the block of the contract-creation transaction | With `startBlock: 0`, the Subgraph scans the full chain history before it finds the first event ([Goldsky](https://docs.goldsky.com/subgraphs/guides/performance)) | Any | +| `endBlock` | Set it for a contract that is retired or replaced | The data source stops processing after that block ([advanced features](/subgraphs/developing/creating/advanced/)) | 0.0.9 | +| Handler type | Use event handlers. Use call handlers and block handlers only when no event exists | Event handlers are far more performant and work on every EVM chain | Any | +| Block handler filters | Never use a block handler with no filter. Use `polling` (every N blocks) or `once` (initialization) | A block handler with no filter runs on every block | 0.0.8 | +| Topic filters (`topic1` to `topic3`) | Filter events on indexed arguments, such as specific addresses | The handler runs only for matching events | 1.2.0 | +| `receipt: true` | Turn it on only for the handlers that need the transaction receipt | Receipts add data to fetch. Use them to read related logs in the same transaction instead of making calls ([Messari](https://github.com/messari/subgraphs/blob/master/docs/PERFORMANCE.md)) | 0.0.5 | +| Unused handlers | Remove handlers for events the schema does not use | Each trigger costs a handler run | Any | + +**Call handler limits:** call handlers and the `call` block filter depend on the Parity tracing API. Some networks do not support it, including BNB Chain and Arbitrum (the docs also name Polygon and Optimism for the `call` filter). A Subgraph that uses them on those networks does not start syncing. + +**Topic filter logic:** values inside one topic are OR ([topic filters](/subgraphs/developing/creating/advanced/)). Different topics are AND. + +```yaml +eventHandlers: + - event: Transfer(indexed address,indexed address,uint256) + handler: handleTransfer + topic1: ['0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045'] # only transfers from this address +blockHandlers: + - handler: handleSnapshot + filter: + kind: polling + every: 100 +``` + +**Handler order:** In each block, event and call handlers run by transaction index. Inside one transaction, event handlers run first, then call handlers, in manifest order. Block handlers run last ([Developer FAQ](/subgraphs/developing/developer-faq/)). Handlers of a new dynamic data source start after all existing handlers for that block. + +**Local testing tip:** set `startBlock` to a recent block for fast feedback. Restore the real start block before you publish ([Graph Node Dev Mode](/subgraphs/developing/creating/graph-node-dev/)). + +## Subgraph Schema Design + +Graph Node, the indexing engine under a Subgraph, stores every entity version in PostgreSQL so that it can handle reorgs and serve Time Travel Queries. Every update writes a new row. Schema choices that cut rows and row size speed up both indexing and queries. + +### Immutable Entities + +Add `(immutable: true)` to [entities that never change after creation](/subgraphs/best-practices/immutable-entities-bytes-as-ids/), such as one entity per onchain event. Graph Node then skips block-range updates and can use database indexes that are [cheaper to build and maintain](https://thegraph.com/blog/two-simple-subgraph-performance-improvements/). + +```graphql +type Transfer @entity(immutable: true) { + id: Bytes! + from: Bytes! + to: Bytes! + value: BigInt! +} +``` + +Do not make an entity immutable if any field changes over time, such as a `status` field. A write to an immutable entity causes an indexing error. + +### Bytes as IDs + +Use `Bytes` for IDs. [Strings take about twice the storage](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) of byte strings for binary data, and string comparison is locale-aware, which costs much more than bytewise comparison. + +```typescript +// Slow: builds a String ID +let id = event.transaction.hash.toHexString().concat('-').concat(event.logIndex.toString()) + +// Fast: builds a Bytes ID +let id = event.transaction.hash.concatI32(event.logIndex.toI32()) + +// Day bucket ID +let id = Bytes.fromI32(event.block.timestamp.toI32() / 86400) + +// Address constants +const BEEF_ADDRESS = Bytes.fromHexString('0xdead...beef') +``` + +- Remove `toHexString()` calls where the ID was an address or hash. +- `Bytes` IDs sort in hex order. To sort by a number, add a `BigInt` field (such as `tokenId`) and order by it. +- Keep `String` IDs only if IDs must be human-readable, or if an external data model needs them. + +### @derivedFrom Instead of Growing Arrays + +An array field is copied in full on every update, so an array that grows bloats the database fast. The [blog guidance](https://thegraph.com/blog/improve-subgraph-performance-avoiding-large-arrays/) is that arrays under about 50 entries are fine; larger arrays that change often are not. Store the link on the child and [derive the array on the parent](/subgraphs/best-practices/derivedfrom/). + +```graphql +type Post @entity { + id: Bytes! + comments: [Comment!]! @derivedFrom(field: "post") +} + +type Comment @entity { + id: Bytes! + content: String! + post: Post! +} +``` + +- Derived fields are read-only in mappings. Use [`loadRelated()`](/subgraphs/developing/creating/graph-ts/api/) to read them in a handler (graph-node 0.31.0, graph-ts 0.31.0, graph-cli 0.51.0 or later). + +### Timeseries and Aggregations + +For time-bucketed metrics (hourly volume, daily totals), [let the database compute aggregates](/subgraphs/best-practices/timeseries/). Requires `specVersion` 1.1.0. + +```graphql +type Data @entity(timeseries: true) { + id: Int8! + timestamp: Timestamp! + price: BigDecimal! +} + +type Stats @aggregation(intervals: ["hour", "day"], source: "Data") { + id: Int8! + timestamp: Timestamp! + sum: BigDecimal! @aggregate(fn: "sum", arg: "price") +} +``` + +- Timeseries entities are always immutable. Graph Node sets `id` and `timestamp`. +- Functions: `sum`, `count`, `min`, `max`, `first`, `last`. `count` supports `cumulative: true`. +- `arg` can be an expression, for example a product of two fields, `greatest`, `least`, `coalesce`, or `case when`. +- Non-aggregated fields (such as a token reference) are dimensions. Aggregates are computed per dimension value. +- Query timestamp filters use microseconds since epoch, not milliseconds. Pass `current` to include the open interval. + +### Types and Query-Friendly Modeling + +- **Sort by numbers, not strings:** Store amounts as `Int` or `BigInt`. String order puts "10" before "2". +- **Put sort keys on the entity itself:** GraphQL queries cannot order by a nested field, so add a `createdAt` or similar field where you sort. +- **Store facts, compute state:** Do not store a value that changes with time but has no event, such as a "LOCKED" status after a lock period. Store the timestamps and durations; [compute the state in the app](https://thegraph.com/blog/subgraph-devex-tips/). +- **Keep counters current:** Fields such as `positionCount` let apps paginate without counting. Update every affected entity in the same handler. +- **Avoid deep nesting:** Give frequently read data a direct path, not a chain of five nested relationships. +- **Use enums for fixed categories:** A field with a known set of values (marketplace, status, order type) should be an [enum](/subgraphs/best-practices/enums/), not a free-form `String`. Enums reject invalid values and typos. + +## Mapping Subgraph Efficiency + +Handler cost multiplies by event count. A small waste in a handler that runs millions of times adds hours to a sync. An `eth_call` is [the most expensive thing a handler can do](/subgraphs/best-practices/avoid-eth-calls/). + +### Why `eth_calls` are Slow + +- [Each call is an RPC round trip](https://thegraph.com/blog/improve-subgraph-performance-reduce-eth-calls/) with a typical call at 100 ms to several seconds. +- Graph Node adds the block hash to each call (EIP-1898) so that results stay correct across reorgs. This makes the call harder for the node to resolve. +- Undeclared calls run one at a time, during handler execution. +- To find calls in a codebase, search for `.bind(`. + +### Remove or Reduce Calls + +1. **Read from the event:** Check `event.params` before you add any call. Values such as reserves are often already in the event. +2. **Cache values that do not change:** Fetch `name`, `symbol`, and `decimals` once, when the entity is first created, then [reuse the stored value](https://docs.goldsky.com/subgraphs/guides/performance). +3. **Compute in the Subgraph:** Derive values such as `totalSupply` from mint and burn events instead of calling the contract. +4. **Declare the calls you keep:** Declared calls run in parallel before the handler, and the handler reads the result from an in-memory cache. + +```typescript +function getOrCreateToken(address: Address): Token { + let token = Token.load(address) + if (token == null) { + token = new Token(address) + let contract = ERC20.bind(address) + let symbol = contract.try_symbol() + token.symbol = symbol.reverted ? '???' : symbol.value + let decimals = contract.try_decimals() + token.decimals = decimals.reverted ? 18 : decimals.value + token.save() + } + return token +} +``` + +### Declared `eth_calls` + +Requires `specVersion` 1.2.0. + +```yaml +eventHandlers: + - event: TransferWithPool(address indexed, address indexed, uint256, bytes32 indexed) + handler: handleTransferWithPool + calls: + poolInfo: ERC20[event.address].getPoolInfo(event.params.to) +``` + +- The label before the colon appears only in error messages. +- The format after the colon is `Contract[address].function(params)`. `address` and `params` can be `event.address` or `event.params.`. +- In the handler, bind the contract and make the call as usual. Graph Node returns the cached result ([Goldsky guide to declared calls](https://docs.goldsky.com/subgraphs/guides/declared-eth-calls)). +- [Declared calls](/subgraphs/developing/creating/advanced/) work only with event handlers. They reduce call time but do not remove it. + +### Entity Reads and Writes + +- **Save only what changed:** Do not load and save entities that the handler does not modify. +- **Use `loadInBlock()` for entities created earlier in the same block:** It skips the database round trip for a [lookup that would miss](/subgraphs/developing/creating/graph-ts/api/). It returns `null` if the entity was not created in the current block, even if it exists in the store. +- **Do not overwrite fresh data with a stale copy:** If a helper loads and saves an entity, reload it in the caller before you save. [Subgraph Linter](/subgraphs/tooling/subgraph-linter/) flags this as `entity-overwrite`. +- **Update every affected entity in one handler:** A stake event can change the protocol total, the user total, and the position. Update all three so that [totals stay in sync](https://thegraph.com/blog/subgraph-devex-tips/). +- **Use `try_` for calls that can revert:** A reverting `decimals()` or `symbol()` is one of the most common causes of a crashed Subgraph. Set a default value and keep indexing. + +## Subgraph Storage and Queries + +### Prune History You Do Not Query + +[Pruning](/subgraphs/best-practices/pruning/) removes old entity versions up to a given block. A smaller database answers queries faster. Set it with `indexerHints` (requires `specVersion` 1.0.0). + +```yaml +specVersion: 1.3.0 +schema: + file: ./schema.graphql +indexerHints: + prune: auto +``` + +| Option | Keeps | Use when | +| --- | --- | --- | +| `auto` | The minimum history the Indexer sets | Default for graph-cli 0.66.0 and later. Best for most app-facing Subgraphs | +| `` | That many recent blocks | You need Time Travel Queries or grafting over a known window | +| `never` | Full history | Default when `indexerHints` is absent. You need Time Travel Queries over all history | + +- You cannot graft at a pruned block. If you graft often, keep a block count that covers your window (the docs suggest about six months). +- `prune: auto` is not recommended with Time Travel Queries. + +### Write Efficient Queries + +- **Use static query strings with variables.** [Do not build query strings at runtime](/subgraphs/querying/best-practices/). Variables let the server cache, sanitize, and statically analyze queries. +- **Request only the fields you use.** Use `@include(if: ...)` and `@skip(if: ...)` for optional fields. +- **Paginate large lists.** graph-client supports automatic pagination and block tracking. +- **Fetch related data in one query.** A direct entity relationship avoids an [N+1 pattern](https://thegraph.com/blog/subgraph-devex-tips/) of one query per item. +- **Keep your API key out of the browser.** Send queries from a server, for example with [Next.js server components](/subgraphs/best-practices/secure-api-keys-nextjs/), and restrict the key to specific Subgraphs and domains in Subgraph Studio. + +## Speeding Up Sync Time and Iteration + +[Find the bottleneck before you change code](https://blog.ormilabs.com/why-subgraph-syncing-is-slow/). A Subgraph [processes blocks in sequence](https://www.streamingfast.io/blog/common-questions/why-is-my-subgraph-syncing-slow), so tuning reduces the work per block but cannot remove the walk through history. + +### Diagnose the Bottleneck + +| Symptom | Likely cause | First fix | +| --- | --- | --- | +| Sync rate is low and constant | Structural: a large block range processed in sequence | Raise `startBlock`, or move extraction to Substreams | +| Sync rate drops sharply at certain blocks | Specific activity triggers contract calls or heavy writes | Fix that handler: cache, declare, or remove calls | +| Slow from the start, even with light handlers | `startBlock` too early, or a block handler with no filter | Set the deployment block; add a `polling` filter | +| Slow on every Subgraph on the same chain | RPC or provider latency | Use a faster endpoint (self-hosted) or check provider status | +| First sync slow, later syncs faster | Cold environment: no shared block or call cache yet | Expected. Judge speed on a warm re-sync | +| Published on The Graph Network but not syncing at all | No Indexer has picked up the Subgraph | Add curation signal to attract Indexers | + +### Iterate Faster During Development + +- **Use a recent `startBlock` locally:** Restore the real block before you publish ([Graph Node Dev Mode](/subgraphs/developing/creating/graph-node-dev/)). +- **Run Graph Node in watch mode:** `gnd dev --watch` rebuilds when files change. Always pass `--node` to `gnd deploy`; without it, `gnd deploy` targets Subgraph Studio. +- **Debug with forking:** Point a local Graph Node at a remote deployment that failed at block X. The local node fetches entities from the remote store, so you [reproduce the failure without syncing to block X](/subgraphs/best-practices/subgraph-debug-forking/). +- **Use operator staging for full-history checks:** Some Gateway Operators offer a staging environment with logs, for syncs too large to run locally. +- **Expect faster re-syncs on hosted providers:** Some providers cache `eth_call` and log results, so a second sync of the same contracts reuses them ([Goldsky](https://docs.goldsky.com/subgraphs/guides/performance) documents a permanent RPC cache). + +### Grafting + +Grafting starts a new deployment from an existing deployment's data at a chosen block, so [a fix does not need a full re-index](/subgraphs/best-practices/grafting/). + +1. Find the last block that indexed correctly. +2. Build the fix. Add `grafting` under `features`. +3. Set `graft.base` to the failed deployment's Deployment ID (not its Subgraph ID) and `graft.block` to that block. +4. Deploy, confirm indexing resumes at the graft point, and confirm the fix. +5. Republish a non-grafted version once it is stable, and move all consumers to it. + +```yaml +features: + - grafting +graft: + base: Qm... # Deployment ID of the failed deployment + block: 6000000 +``` + +- **Graft-safe:** changes inside mappings or the manifest, such as a fixed handler or a removed handler. +- **Not graft-safe:** changing or removing field types, and large mapping rewrites. Schema changes such as immutable entities or `Bytes` IDs also break graft compatibility. +- **Grafting does not make later blocks faster.** [Remaining blocks index faster only if the new code does less work per event](https://docs.goldsky.com/subgraphs/guides/performance). +- **Not for The Graph Network.** The docs do not recommend grafting for mainnet Subgraphs on the network. Some Indexers may not support it. +- **Pruning conflict:** you cannot graft at a pruned block. + +Grafting also works when a contract is redeployed at a new address: the new Subgraph indexes the new contract and keeps the old contract's history. The [Grafting](/subgraphs/best-practices/grafting/) page walks through both scenarios. + +### When the Design is the Limit + +- **Rebuild lean and cut over.** If the fix needs schema changes, deploy a leaner version under a new name, keep the original serving, and switch after the new one syncs. Keep live-state entities and raw events; drop per-event pricing, unused metadata, and per-event calls. +- **Split it with Subgraph composition.** Build small source Subgraphs that each index one set of events, then [compose them](/subgraphs/best-practices/subgraph-composition/) into one Subgraph for your app. Each source syncs and resyncs on its own. +- **Use a Substreams-powered Subgraph.** Substreams processes history in parallel, then streams live blocks. The schema and query interface can stay the same. [The Graph blog](https://thegraph.com/blog/substreams-powered-subgraphs/) reports one case that went from 1,440 hours to 20 hours. + +## Subgraph Reliability and Testing + +A crash at block 15 million costs a full re-sync or a graft. Catch bugs before you deploy. + +- **Run Subgraph Linter:** It is a [static analysis tool](/subgraphs/tooling/subgraph-linter/), as a CLI or a VS Code extension. Its checks include `entity-overwrite`, `unexpected-null`, `unchecked-load`, `division-guard`, `derived-field-guard`, and `undeclared-eth-call`. Only checks set to `error` fail a CI run. +- **Write unit tests with Matchstick:** [Mock events, contract calls](/subgraphs/tooling/unit-testing-framework/) (including reverts), IPFS files, and `loadInBlock` results. +- **Handle missing entities:** Do not force-unwrap `Entity.load(...)` with `!`. Handle the case where the entity does not exist yet. +- **Guard divisions and reverts.** Check for a zero denominator. Use `try_` calls with defaults. +- **Know the limits of non-fatal errors:** The `nonFatalErrors` feature lets a synced Subgraph skip a handler that fails deterministically. [The Graph Network does not support it](/subgraphs/developing/creating/advanced/), so do not publish to the network with it. +- **Version deliberately:** Deploy a contract upgrade or schema change as a new version, not an edit to the live one. Keep the old version serving until the new one syncs. + +## Move an Existing Subgraph to The Graph + +A Subgraph that runs on another platform can move to The Graph Network without code changes. Deploy it to Subgraph Studio from its source code or its IPFS hash, publish it, and add curation signal so that Indexers pick it up. Apps keep the same schema and queries; only the query URL changes. See [Transfer to The Graph](/subgraphs/best-practices/transfer-to-the-graph/). + +## Benchmarks and Evidence + +In a [controlled test from April 2022](https://thegraph.com/blog/two-simple-subgraph-performance-improvements/), using immutable entities and `Bytes` IDs together cut time per block by 28%, sync time by 26%, and storage by 48%. The test compared four variants of one Subgraph. + +| Variant | Avg time per block (ms) | Sync time (hours) | Storage (GB) | Storage reduction | +| ---------------------------- | ----------------------- | ----------------- | ------------ | ----------------- | +| Base (`ID` strings, mutable) | 268 | 352 | 143 | none | +| Immutable entities | 217 | 292 | 89 | 37% | +| `Bytes` IDs | 225 | 299 | 115 | 20% | +| Both | 194 | 259 | 74 | 48% | + +**Other measured findings (Messari, Uniswap v2 Subgraph):** + +- Adding hourly metrics entities changed speed by less than 0.2% (4.5646 vs. 4.5724 blocks per second). The author named event and call handlers as the main bottleneck. +- A price oracle that made a Chainlink call per event was about 35% slower. After skipping the call before the feed existed and pricing each token once per block, the gap fell to about 4%. + +**Results vary by workload.** Gains depend on how much time a Subgraph spends on database writes compared to contract calls and reads. Test changes against your own Subgraph before and after. + +## Sources + +**The Graph docs: this section** + +- [Improve query speed with Subgraph pruning](/subgraphs/best-practices/pruning/) +- [Use @derivedFrom for arrays](/subgraphs/best-practices/derivedfrom/) +- [Immutable entities and Bytes as IDs](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) +- [Avoid eth_calls](/subgraphs/best-practices/avoid-eth-calls/) +- [Timeseries and aggregations](/subgraphs/best-practices/timeseries/) +- [Graft a Subgraph to ship fixes and replace contracts](/subgraphs/best-practices/grafting/) +- [Debug with Subgraph forking](/subgraphs/best-practices/subgraph-debug-forking/) + +**The Graph docs: related pages** + +- [Subgraph manifest](/subgraphs/developing/creating/subgraph-manifest/) +- [Advanced Subgraph features](/subgraphs/developing/creating/advanced/) +- [graph-ts API](/subgraphs/developing/creating/graph-ts/api/) +- [Developer FAQ](/subgraphs/developing/developer-faq/) +- [Graph Node Dev Mode](/subgraphs/developing/creating/graph-node-dev/) +- [Subgraph Linter](/subgraphs/tooling/subgraph-linter/) +- [Unit testing framework](/subgraphs/tooling/unit-testing-framework/) +- [Querying best practices](/subgraphs/querying/best-practices/) + +**The Graph blog** + +- [Improve Subgraph indexing performance easily: reduce eth_calls](https://thegraph.com/blog/improve-subgraph-performance-reduce-eth-calls/) +- [Best practices in Subgraph development: avoiding large arrays](https://thegraph.com/blog/improve-subgraph-performance-avoiding-large-arrays/) +- [Two simple Subgraph performance improvements](https://thegraph.com/blog/two-simple-subgraph-performance-improvements/) +- [9 time and money saving Subgraph optimizations](https://thegraph.com/blog/subgraph-optimization-tips/) +- [The Subgraph DevEx Blueprint](https://thegraph.com/blog/subgraph-devex-tips/) +- [Boost indexing performance with Substreams-powered Subgraphs](https://thegraph.com/blog/substreams-powered-subgraphs/) diff --git a/website/src/pages/en/subgraphs/best-practices/pruning.mdx b/website/src/pages/en/subgraphs/best-practices/pruning.mdx index 6f7ea4903d3d..36d736fefe04 100644 --- a/website/src/pages/en/subgraphs/best-practices/pruning.mdx +++ b/website/src/pages/en/subgraphs/best-practices/pruning.mdx @@ -1,11 +1,8 @@ --- -title: Subgraph Best Practice 1 - Improve Query Speed with Subgraph Pruning -sidebarTitle: 'Pruning with indexerHints' +title: Improve Query Speed with Pruning --- -## TLDR - -[Pruning](/developing/creating-a-subgraph/#prune) removes archival entities from the Subgraph’s database up to a given block, and removing unused entities from a Subgraph’s database will improve a Subgraph’s query performance, often dramatically. Using `indexerHints` is an easy way to prune a Subgraph. +[Pruning](/subgraphs/developing/creating/subgraph-manifest/#prune) removes old entity versions from a Subgraph's database up to a given block. A smaller database answers queries faster, often dramatically. Setting `indexerHints` in the manifest is the simplest way to prune. ## How to Prune a Subgraph With `indexerHints` @@ -15,9 +12,9 @@ Add a section called `indexerHints` in the manifest. - `prune: auto`: Retains the minimum necessary history as set by the Indexer, optimizing query performance. This is the generally recommended setting and is the default for all Subgraphs created by `graph-cli` >= 0.66.0. - `prune: `: Sets a custom limit on the number of historical blocks to retain. -- `prune: never`: No pruning of historical data; retains the entire history and is the default if there is no `indexerHints` section. `prune: never` should be selected if [Time Travel Queries](/subgraphs/querying/graphql-api/#time-travel-queries) are desired. +- `prune: never`: No pruning of historical data; retains the entire history and is the default if there is no `indexerHints` section. `prune: never` should be selected if [Time Travel Queries](/subgraphs/querying/graphql-api/#time-travel-queries-example) are desired. -We can add `indexerHints` to our Subgraphs by updating our `subgraph.yaml`: +Add `indexerHints` to your `subgraph.yaml` (requires `specVersion` 1.0.0 or later): ```yaml specVersion: 1.3.0 @@ -33,24 +30,12 @@ dataSources: ## Important Considerations -- If [Time Travel Queries](/subgraphs/querying/graphql-api/#time-travel-queries) are desired as well as pruning, pruning must be performed accurately to retain Time Travel Query functionality. Due to this, it is generally not recommended to use `indexerHints: prune: auto` with Time Travel Queries. Instead, prune using `indexerHints: prune: ` to accurately prune to a block height that preserves the historical data required by Time Travel Queries, or use `prune: never` to maintain all data. - -- It is not possible to [graft](/subgraphs/cookbook/grafting/) at a block height that has been pruned. If grafting is routinely performed and pruning is desired, it is recommended to use `indexerHints: prune: ` that will accurately retain a set number of blocks (e.g., enough for six months). - -## Conclusion - -Pruning using `indexerHints` is a best practice for Subgraph development, offering significant query performance improvements. - -## Subgraph Best Practices 1-6 - -1. [Improve Query Speed with Subgraph Pruning](/subgraphs/best-practices/pruning/) - -2. [Improve Indexing and Query Responsiveness by Using @derivedFrom](/subgraphs/best-practices/derivedfrom/) - -3. [Improve Indexing and Query Performance by Using Immutable Entities and Bytes as IDs](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) +- If [Time Travel Queries](/subgraphs/querying/graphql-api/#time-travel-queries-example) are desired as well as pruning, pruning must be performed accurately to retain Time Travel Query functionality. Due to this, it is generally not recommended to use `indexerHints: prune: auto` with Time Travel Queries. Instead, prune using `indexerHints: prune: ` to accurately prune to a block height that preserves the historical data required by Time Travel Queries, or use `prune: never` to maintain all data. -4. [Improve Indexing Speed by Avoiding `eth_calls`](/subgraphs/best-practices/avoid-eth-calls/) +- It is not possible to [graft](/subgraphs/best-practices/grafting/) at a block height that has been pruned. If grafting is routinely performed and pruning is desired, it is recommended to use `indexerHints: prune: ` that will accurately retain a set number of blocks (e.g., enough for six months). -5. [Simplify and Optimize with Timeseries and Aggregations](/subgraphs/best-practices/timeseries/) +## Learn More -6. [Use Grafting for Quick Hotfix Deployment](/subgraphs/best-practices/grafting-hotfix/) +- [Time Travel Queries](/subgraphs/querying/graphql-api/#time-travel-queries-example): query entity state at a past block. +- [Grafting](/subgraphs/best-practices/grafting/): why your pruning window matters if you graft. +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the full checklist and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/guides/secure-api-keys-nextjs.mdx b/website/src/pages/en/subgraphs/best-practices/secure-api-keys-nextjs.mdx similarity index 64% rename from website/src/pages/en/subgraphs/guides/secure-api-keys-nextjs.mdx rename to website/src/pages/en/subgraphs/best-practices/secure-api-keys-nextjs.mdx index d9fda08a321c..aca7fa9192ab 100644 --- a/website/src/pages/en/subgraphs/guides/secure-api-keys-nextjs.mdx +++ b/website/src/pages/en/subgraphs/best-practices/secure-api-keys-nextjs.mdx @@ -1,12 +1,10 @@ --- -title: How to Secure API Keys Using Next.js Server Components +title: Secure API Keys with Next.js Server Components --- -## Overview +An API key in frontend code is visible to anyone who opens the browser's developer tools. Use [Next.js server components](https://nextjs.org/docs/app/building-your-application/rendering/server-components) to send Subgraph queries from the server, so the key never reaches the client. For more protection, also [restrict your API key to specific Subgraphs or domains in Subgraph Studio](/subgraphs/providers/subgraph-studio/managing-api-keys/). -We can use [Next.js server components](https://nextjs.org/docs/app/building-your-application/rendering/server-components) to properly secure our API key from exposure in the frontend of our dapp. To further increase our API key security, we can also [restrict our API key to certain Subgraphs or domains in Subgraph Studio](/cookbook/upgrading-a-subgraph/#securing-your-api-key). - -In this cookbook, we will go over how to create a Next.js server component that queries a Subgraph while also hiding the API key from the frontend. +This guide shows how to create a Next.js server component that queries a Subgraph while keeping the API key out of the frontend. ### Caveats @@ -28,16 +26,16 @@ In a standard React application, API keys included in the frontend code can be e - Basic knowledge of Next.js and React. - An existing Next.js project that uses the [App Router](https://nextjs.org/docs/app). -## Step-by-Step Cookbook +## Step-by-Step Guide ### Step 1: Set Up Environment Variables -1. In our Next.js project root, create a `.env.local` file. -2. Add our API key: `API_KEY=`. +1. In your Next.js project root, create a `.env.local` file. +2. Add your API key: `API_KEY=`. ### Step 2: Create a Server Component -1. In our `components` directory, create a new file, `ServerComponent.js`. +1. In your `components` directory, create a new file, `ServerComponent.js`. 2. Use the provided example code to set up the server component. ### Step 3: Implement Server-Side API Request @@ -97,7 +95,7 @@ export default async function ServerComponent() { ### Step 4: Use the Server Component -1. In our page file (e.g., `pages/index.js`), import `ServerComponent`. +1. In your page file (e.g., `pages/index.js`), import `ServerComponent`. 2. Render the component: ```javascript @@ -112,12 +110,16 @@ export default function Home() { } ``` -### Step 5: Run and Test Our Dapp +### Step 5: Run and Test Your App -Start our Next.js application using `npm run dev`. Verify that the server component is fetching data without exposing the API key. +Start your Next.js application using `npm run dev`. Verify that the server component is fetching data without exposing the API key. ![Server-side rendering](/img/api-key-server-side-rendering.png) -### Conclusion +The API key now stays on the server, and the browser receives only the query results. + +## Learn More -By utilizing Next.js Server Components, we've effectively hidden the API key from the client-side, enhancing the security of our application. This method ensures that sensitive operations are handled server-side, away from potential client-side vulnerabilities. Finally, be sure to explore [other API key security measures](/subgraphs/providers/subgraph-studio/managing-api-keys/) to increase your API key security even further. +- [Managing API keys](/subgraphs/providers/subgraph-studio/managing-api-keys/): restrict keys by Subgraph and domain, and set spending limits. +- [Querying best practices](/subgraphs/querying/best-practices/): write efficient, cacheable queries. +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the full checklist and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/guides/subgraph-composition.mdx b/website/src/pages/en/subgraphs/best-practices/subgraph-composition.mdx similarity index 74% rename from website/src/pages/en/subgraphs/guides/subgraph-composition.mdx rename to website/src/pages/en/subgraphs/best-practices/subgraph-composition.mdx index 25b4bf00c2df..201eed2e7d3f 100644 --- a/website/src/pages/en/subgraphs/guides/subgraph-composition.mdx +++ b/website/src/pages/en/subgraphs/best-practices/subgraph-composition.mdx @@ -1,15 +1,8 @@ --- -title: Aggregate Data Using Subgraph Composition -sidebarTitle: 'Build a Composable Subgraph with Multiple Subgraphs' +title: Combine Data with Subgraph Composition --- -Leverage Subgraph composition to speed up development time. Create a base Subgraph with essential data, then build additional Subgraphs on top of it. - -Optimize your Subgraph by merging data from independent, source Subgraphs into a single composable Subgraph to enhance data aggregation. - -## Introduction - -Composable Subgraphs enable you to combine multiple Subgraphs' data sources into a new Subgraph, facilitating faster and more flexible Subgraph development. Subgraph composition empowers you to create and maintain smaller, focused Subgraphs that collectively form a larger, interconnected dataset. +Subgraph composition lets you build a Subgraph on top of other Subgraphs. Instead of one large Subgraph that indexes everything, you maintain small, focused source Subgraphs, then combine their data into a single composed Subgraph for your app. Each source syncs and resyncs on its own, which speeds up development and recovery. ### Benefits of Composition @@ -40,11 +33,11 @@ While the source Subgraph is a standard Subgraph, the dependent Subgraph uses th ### Source Subgraphs - All Subgraphs need to be published with a **specVersion 1.3.0 or later** (Use the latest graph-cli version to be able to deploy composable Subgraphs). -- See notes here: https://github.com/graphprotocol/graph-node/releases/tag/v0.37.0 +- See the [Graph Node v0.37.0 release notes](https://github.com/graphprotocol/graph-node/releases/tag/v0.37.0). - Immutable entities only: All Subgraphs must have [immutable entities](/subgraphs/best-practices/immutable-entities-bytes-as-ids/#immutable-entities) when the Subgraph is deployed. - Pruning can be used in the source Subgraphs, but only entities that are immutable can be composed on top of. - Source Subgraphs cannot use grafting on top of existing entities. -- Aggregated entities can be used in composition, but entities that are composed from them cannot performed additional aggregations directly. +- Aggregated entities can be used in composition, but entities that are composed from them cannot perform additional aggregations directly. ### Composed Subgraphs @@ -52,13 +45,13 @@ While the source Subgraph is a standard Subgraph, the dependent Subgraph uses th - Composed Subgraphs can only use **datasources from the same chain.** - **Nested composition is not yet supported**: Composing on top of another composed Subgraph isn’t allowed at this time. - Aggregated entities can be used in composition, but the composed entities on them cannot also use aggregations directly. -- Developers cannot compose an onchain datasource with a Subgraph datasource (i.e. you can’t do normal event handlers and call handlers and block handlers in a composed Subgraph). +- You cannot combine an onchain data source with a Subgraph data source (i.e. you can’t do normal event handlers and call handlers and block handlers in a composed Subgraph). -Additionally, you can explore the [example-composable-subgraph](https://github.com/graphprotocol/example-composable-subgraph) repository for a working implementation of composable Subgraphs +Additionally, you can explore the [example-composable-subgraph](https://github.com/graphprotocol/example-composable-subgraph) repository for a working implementation of composable Subgraphs. ## Get Started -The following guide provides examples for defining 3 source Subgraphs to create one powerful composed Subgraph. +The following steps define three source Subgraphs and combine them into one composed Subgraph. ### Specifics @@ -121,12 +114,13 @@ This composed Subgraph combines and aggregates the information from the source S ## Key Takeaways -- This powerful tool will scale your Subgraph development and allow you to combine multiple Subgraphs. -- The setup includes the deployment of 3 source Subgraphs and one final deployment of the composed Subgraph. -- This feature unlocks scalability, simplifying both development and maintenance efficiency. +- Composition lets you combine up to five source Subgraphs on the same chain into one Subgraph. +- The setup deploys three source Subgraphs first, then the composed Subgraph. +- Small, focused source Subgraphs are easier to develop, maintain, and resync than one large Subgraph. -## Additional Resources +## Learn More - Check out all the code for this example in [this GitHub repo](https://github.com/graphprotocol/example-composable-subgraph). -- To add advanced features to your Subgraph, check out [Subgraph advanced features](/developing/creating/advanced/). -- To learn more about aggregations, check out [Timeseries and Aggregations](/subgraphs/developing/creating/advanced/#timeseries-and-aggregations). +- To add advanced features to your Subgraph, check out [Advanced Subgraph Features](/subgraphs/developing/creating/advanced/). +- To learn more about aggregations, check out [Timeseries and Aggregations](/subgraphs/best-practices/timeseries/). +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the full checklist and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/best-practices/subgraph-debug-forking.mdx b/website/src/pages/en/subgraphs/best-practices/subgraph-debug-forking.mdx new file mode 100644 index 000000000000..8c7f1c316bc3 --- /dev/null +++ b/website/src/pages/en/subgraphs/best-practices/subgraph-debug-forking.mdx @@ -0,0 +1,100 @@ +--- +title: Debug Faster with Subgraph Forking +--- + +Syncing a Subgraph to the block where it failed can take hours, and every attempted fix means another wait. **Subgraph forking**, developed by [LimeChain](https://limechain.tech/), removes that wait: you reproduce and fix the failure locally without syncing to the failing block. This guide shows how to use it. + +## What Subgraph Forking Is + +**Subgraph forking** is the process of lazily fetching entities from _another_ Subgraph's store (usually a remote one). + +In the context of debugging, **Subgraph forking** allows you to debug your failed Subgraph at block _X_ without needing to wait to sync-up to block _X_. + +## How It Works + +When you deploy a Subgraph to a remote Graph Node for indexing and it fails at block _X_, the good news is that the Graph Node will still serve GraphQL queries using its store, which is synced-up to block _X_. You can use this up-to-date store to fix the bug that occurs at block _X_. + +In short, you _fork the failing Subgraph_ from a remote Graph Node that is guaranteed to have the Subgraph indexed up to block _X_ in order to provide the locally deployed Subgraph being debugged at block _X_ an up-to-date view of the indexing state. + +## Example: Fix a Failing Gravatar Subgraph + +To stay focused on debugging, this example uses the [example-Subgraph](https://github.com/graphprotocol/graph-tooling/tree/main/examples/ethereum-gravatar) indexing the Ethereum Gravity smart contract. + +Here are the handlers that index `Gravatar`s. They look correct: + +```tsx +export function handleNewGravatar(event: NewGravatar): void { + let gravatar = new Gravatar(event.params.id.toHex().toString()) + gravatar.owner = event.params.owner + gravatar.displayName = event.params.displayName + gravatar.imageUrl = event.params.imageUrl + gravatar.save() +} + +export function handleUpdatedGravatar(event: UpdatedGravatar): void { + let gravatar = Gravatar.load(event.params.id.toI32().toString()) + if (gravatar == null) { + log.critical('Gravatar not found!', []) + return + } + gravatar.owner = event.params.owner + gravatar.displayName = event.params.displayName + gravatar.imageUrl = event.params.imageUrl + gravatar.save() +} +``` + +When you deploy this Subgraph to [Subgraph Studio](https://thegraph.com/studio/), it fails with the _"Gravatar not found!"_ error. + +The usual way to attempt a fix is: + +1. Make a change in the mappings that you believe will solve the issue. +2. Re-deploy the Subgraph to [Subgraph Studio](https://thegraph.com/studio/) (or another remote Graph Node). +3. Wait for it to sync. +4. If it breaks again, go back to step 1. + +Step 3 is what makes this slow. **Subgraph forking** removes it: + +0. Spin-up a local Graph Node with the **_appropriate fork-base_** set. +1. Make a change in the mappings source, which you believe will solve the issue. +2. Deploy to the local Graph Node, **_forking the failing Subgraph_** and **_starting from the problematic block_**. +3. If it breaks again, go back to step 1. + +Two terms need explaining: + +- **`fork-base`** is the base URL of the remote store. When you append a Subgraph ID, the result (`/`) is a valid GraphQL endpoint for that Subgraph's store. +- **Forking the failing Subgraph** means passing its ID to `graph deploy` with `--debug-fork`: + +```bash +$ graph deploy --debug-fork --ipfs http://localhost:5001 --node http://localhost:8020 +``` + +Also set `dataSources.source.startBlock` in the manifest to the problematic block, so the local node skips the blocks before it and reads that state from the fork. + +Here is the full fix for the Gravatar example: + +1. Start a local Graph Node ([setup instructions](https://github.com/graphprotocol/graph-node#running-a-local-graph-node)) with the `fork-base` option set to `https://api.thegraph.com/subgraphs/id/`, so that it can fork the failing Subgraph deployed to [Subgraph Studio](https://thegraph.com/studio/). + +``` +$ cargo run -p graph-node --release -- \ + --postgres-url postgresql://USERNAME[:PASSWORD]@localhost:5432/graph-node \ + --ethereum-rpc NETWORK_NAME:[CAPABILITIES]:URL \ + --ipfs 127.0.0.1:5001 + --fork-base https://api.thegraph.com/subgraphs/id/ +``` + +2. Inspect the handlers. The two handlers use different `id` representations for `Gravatar`s. While `handleNewGravatar` converts it to a hex (`event.params.id.toHex()`), `handleUpdatedGravatar` uses an int32 (`event.params.id.toI32()`) which causes the `handleUpdatedGravatar` to panic with "Gravatar not found!". Change both handlers to convert the `id` to hex. +3. Deploy the fixed Subgraph to the local Graph Node, **_forking the failing Subgraph_** and setting `dataSources.source.startBlock` to `6190343` in `subgraph.yaml`: + +```bash +$ graph deploy gravity --debug-fork QmNp169tKvomnH3cPXTfGg4ZEhAHA6kEq5oy1XDqAxqHmW --ipfs http://localhost:5001 --node http://localhost:8020 +``` + +4. Check the local Graph Node logs to confirm that the handler now succeeds at the problematic block. +5. Deploy the fixed Subgraph to Subgraph Studio or another remote Graph Node. + +## Learn More + +- [Graph Node Dev Mode](/subgraphs/developing/creating/graph-node-dev/): run and iterate on a Subgraph locally. +- [Grafting](/subgraphs/best-practices/grafting/): ship the fix without a full re-index. +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the full checklist and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/best-practices/timeseries.mdx b/website/src/pages/en/subgraphs/best-practices/timeseries.mdx index d745dd5695ec..1f656a726df5 100644 --- a/website/src/pages/en/subgraphs/best-practices/timeseries.mdx +++ b/website/src/pages/en/subgraphs/best-practices/timeseries.mdx @@ -1,15 +1,8 @@ --- -title: Subgraph Best Practice 5 - Simplify and Optimize with Timeseries and Aggregations -sidebarTitle: 'Timeseries and Aggregations' +title: Simplify Metrics with Timeseries and Aggregations --- -## TLDR - -Leveraging the new time-series and aggregations feature in Subgraphs can significantly enhance both indexing speed and query performance. - -## Overview - -Timeseries and aggregations reduce data processing overhead and accelerate queries by offloading aggregation computations to the database and simplifying mapping code. This approach is particularly effective when handling large volumes of time-based data. +For time-bucketed metrics such as hourly volume or daily totals, let the database compute the aggregates. Timeseries and aggregations move that work out of your mappings, which speeds up both indexing and queries and simplifies your code. The benefit grows with the volume of time-based data. ## Benefits of Timeseries and Aggregations @@ -20,7 +13,7 @@ Timeseries and aggregations reduce data processing overhead and accelerate queri 2. Simplified Mapping Code -- No Manual Calculations: Developers no longer need to write complex aggregation logic in mappings. +- No Manual Calculations: You no longer write complex aggregation logic in mappings. - Reduced Complexity: Simplifies code maintenance and minimizes the potential for errors. 3. Dramatically Faster Queries @@ -31,14 +24,14 @@ Timeseries and aggregations reduce data processing overhead and accelerate queri ### Important Considerations - Immutable Data: Timeseries data cannot be altered once written, ensuring data integrity and simplifying indexing. -- Automatic ID and Timestamp Management: id and timestamp fields are automatically managed by graph-node, reducing potential errors. +- Automatic ID and Timestamp Management: Graph Node manages the `id` and `timestamp` fields, reducing potential errors. - Efficient Data Storage: By separating raw data from aggregates, storage is optimized, and queries run faster. ## How to Implement Timeseries and Aggregations ### Prerequisites -You need `spec version 1.1.0` for this feature. +This feature requires `specVersion` 1.1.0 or later. ### Defining Timeseries Entities @@ -157,11 +150,11 @@ Supported aggregation functions: ### Examples of Aggregation Expressions -- Sum Token Value: @aggregate(fn: "sum", arg: "priceUSD \_ amount") -- Maximum Positive Amount: @aggregate(fn: "max", arg: "greatest(amount0, amount1, 0)") -- Conditional Sum: @aggregate(fn: "sum", arg: "case when amount0 > amount1 then amount0 else 0 end") +- Sum Token Value: `@aggregate(fn: "sum", arg: "priceUSD * amount")` +- Maximum Positive Amount: `@aggregate(fn: "max", arg: "greatest(amount0, amount1, 0)")` +- Conditional Sum: `@aggregate(fn: "sum", arg: "case when amount0 > amount1 then amount0 else 0 end")` -Supported operators and functions include basic arithmetic (+, -, \_, /), comparison operators, logical operators (and, or, not), and SQL functions like greatest, least, coalesce, etc. +Supported operators and functions include basic arithmetic (`+`, `-`, `*`, `/`), comparison operators, logical operators (and, or, not), and SQL functions like greatest, least, coalesce, etc. ### Query Parameters @@ -174,26 +167,8 @@ Supported operators and functions include basic arithmetic (+, -, \_, /), compar - Sorting: Results are automatically sorted by timestamp and id in descending order. - Current Data: An optional current argument can include the current, partially filled interval. -### Conclusion - -Implementing timeseries and aggregations in Subgraphs is a best practice for projects dealing with time-based data. This approach: - -- Enhances Performance: Speeds up indexing and querying by reducing data processing overhead. -- Simplifies Development: Eliminates the need for manual aggregation logic in mappings. -- Scales Efficiently: Handles large volumes of data without compromising on speed or responsiveness. - -By adopting this pattern, developers can build more efficient and scalable Subgraphs, providing faster and more reliable data access to end-users. To learn more about implementing timeseries and aggregations, refer to the [Timeseries and Aggregations Readme](https://github.com/graphprotocol/graph-node/blob/master/docs/aggregations.md) and consider experimenting with this feature in your Subgraphs. - -## Subgraph Best Practices 1-6 - -1. [Improve Query Speed with Subgraph Pruning](/subgraphs/best-practices/pruning/) - -2. [Improve Indexing and Query Responsiveness by Using @derivedFrom](/subgraphs/best-practices/derivedfrom/) - -3. [Improve Indexing and Query Performance by Using Immutable Entities and Bytes as IDs](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) - -4. [Improve Indexing Speed by Avoiding `eth_calls`](/subgraphs/best-practices/avoid-eth-calls/) - -5. [Simplify and Optimize with Timeseries and Aggregations](/subgraphs/best-practices/timeseries/) +## Learn More -6. [Use Grafting for Quick Hotfix Deployment](/subgraphs/best-practices/grafting-hotfix/) +- [Timeseries and Aggregations README](https://github.com/graphprotocol/graph-node/blob/master/docs/aggregations.md): the full Graph Node reference. +- [Timeseries and aggregations](/subgraphs/developing/creating/advanced/#timeseries-and-aggregations) in Advanced Subgraph Features. +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the full checklist and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/guides/transfer-to-the-graph.mdx b/website/src/pages/en/subgraphs/best-practices/transfer-to-the-graph.mdx similarity index 73% rename from website/src/pages/en/subgraphs/guides/transfer-to-the-graph.mdx rename to website/src/pages/en/subgraphs/best-practices/transfer-to-the-graph.mdx index a62072c48373..da1ad6d92d6f 100644 --- a/website/src/pages/en/subgraphs/guides/transfer-to-the-graph.mdx +++ b/website/src/pages/en/subgraphs/best-practices/transfer-to-the-graph.mdx @@ -1,8 +1,8 @@ --- -title: Transfer to The Graph +title: Transfer Subgraphs to The Graph --- -Quickly upgrade your Subgraphs from any platform to [The Graph's decentralized network](https://thegraph.com/networks/). +Move a Subgraph from any platform to [The Graph Network](https://thegraph.com/networks/) without changing its code. Apps keep the same schema and queries; only the query URL changes. ## Benefits of Switching to The Graph @@ -10,11 +10,11 @@ Quickly upgrade your Subgraphs from any platform to [The Graph's decentralized n - Increase reliability from a global network supported by 100+ Indexers. - Receive lightning-fast support for Subgraphs 24/7, with an on-call engineering team. -## Upgrade Your Subgraph to The Graph in 3 Easy Steps +## Transfer Your Subgraph in Three Steps -1. [Set Up Your Studio Environment](/subgraphs/cookbook/transfer-to-the-graph/#1-set-up-your-studio-environment) -2. [Deploy Your Subgraph to Studio](/subgraphs/cookbook/transfer-to-the-graph/#2-deploy-your-subgraph-to-studio) -3. [Publish to The Graph Network](/subgraphs/cookbook/transfer-to-the-graph/#publish-your-subgraph-to-the-graphs-decentralized-network) +1. [Set Up Your Studio Environment](#1-set-up-your-studio-environment) +2. [Deploy Your Subgraph to Studio](#2-deploy-your-subgraph-to-studio) +3. [Publish to The Graph Network](#3-publish-your-subgraph-to-the-graph-network) ## 1. Set Up Your Studio Environment @@ -70,7 +70,7 @@ graph deploy --ipfs-hash ### Query Your Subgraph -> To attract about 3 indexers to query your Subgraph, it’s recommended to curate at least 3,000 GRT. To learn more about curating, check out [Curating](/resources/roles/curating/) on The Graph. +> Add curation signal when you publish so that Indexers pick up your Subgraph: at least 1,000 GRT on networks with indexing rewards, or 500 GRT on community-supported networks. To learn more, see [Curating](/resources/roles/curating/). You can start [querying](/subgraphs/querying/introduction/) any Subgraph by sending a GraphQL query into the Subgraph’s query URL endpoint, which is located at the top of its Explorer page in Subgraph Studio. @@ -98,7 +98,8 @@ You can create API Keys in Subgraph Studio under the “API Keys” menu at the Once you upgrade, you can access and manage your Subgraphs in [Subgraph Studio](https://thegraph.com/studio/) and explore all Subgraphs in [The Graph Explorer](https://thegraph.com/networks/). -### Additional Resources +## Learn More -- To quickly create and publish a new Subgraph, check out the [Quick Start](/subgraphs/quick-start/). -- To explore all the ways you can optimize and customize your Subgraph for a better performance, read more about [creating a Subgraph here](/developing/creating-a-subgraph/). +- To create and publish a new Subgraph, check out the [Quick Start](/subgraphs/quick-start/). +- To learn more about attracting Indexers with signal, see [Curating](/resources/roles/curating/). +- [Guides & Best Practices overview](/subgraphs/best-practices/overview/): the checklist to run on your Subgraph after you transfer it, and how this page fits with the rest of the section. diff --git a/website/src/pages/en/subgraphs/developing/creating/advanced.mdx b/website/src/pages/en/subgraphs/developing/creating/advanced.mdx index 0f33c514f8cb..3e7ab163d0c0 100644 --- a/website/src/pages/en/subgraphs/developing/creating/advanced.mdx +++ b/website/src/pages/en/subgraphs/developing/creating/advanced.mdx @@ -531,7 +531,7 @@ calls: ### Grafting onto Existing Subgraphs -> **Note:** it is not recommended to use grafting when initially upgrading to The Graph Network. Learn more [here](/subgraphs/cookbook/grafting/#important-note-on-grafting-when-upgrading-to-the-network). +> **Note:** it is not recommended to use grafting when initially upgrading to The Graph Network. Learn more [here](/subgraphs/best-practices/grafting/#when-not-to-use-grafting). When a Subgraph is first deployed, it starts indexing events at the genesis block of the corresponding chain (or at the `startBlock` defined with each data source) In some circumstances; it is beneficial to reuse the data from an existing Subgraph and start indexing at a much later block. This mode of indexing is called _Grafting_. Grafting is, for example, useful during development to get past simple errors in the mappings quickly or to temporarily get an existing Subgraph working again after it has failed. diff --git a/website/src/pages/en/subgraphs/developing/creating/subgraph-manifest.mdx b/website/src/pages/en/subgraphs/developing/creating/subgraph-manifest.mdx index 6df8dd98d66f..1ad4a3bf5e7c 100644 --- a/website/src/pages/en/subgraphs/developing/creating/subgraph-manifest.mdx +++ b/website/src/pages/en/subgraphs/developing/creating/subgraph-manifest.mdx @@ -539,7 +539,7 @@ indexerHints: | Version | Release notes | | :-: | --- | -| 1.3.0 | Added support for [Subgraph Composition](/cookbook/subgraph-composition-three-sources) | +| 1.3.0 | Added support for [Subgraph Composition](/subgraphs/best-practices/subgraph-composition/) | | 1.2.0 | Added support for [Indexed Argument Filtering](/developing/creating/advanced/#indexed-argument-filters--topic-filters) & declared `eth_call` | | 1.1.0 | Supports [Timeseries & Aggregations](/developing/creating/advanced/#timeseries-and-aggregations). Added support for type `Int8` for `id`. | | 1.0.0 | Supports [`indexerHints`](/developing/creating/subgraph-manifest/#indexer-hints) feature to prune Subgraphs | diff --git a/website/src/pages/en/subgraphs/guides/_meta.js b/website/src/pages/en/subgraphs/guides/_meta.js deleted file mode 100644 index c3ea33cb5a4a..000000000000 --- a/website/src/pages/en/subgraphs/guides/_meta.js +++ /dev/null @@ -1,8 +0,0 @@ -export default { - 'transfer-to-the-graph': 'Transfer to The Graph', - grafting: 'Subgraph Grafting', - 'subgraph-debug-forking': 'Subgraph Forking', - 'subgraph-composition': 'Composable Subgraphs', - enums: 'Using Enums', - 'secure-api-keys-nextjs': 'Securing API Keys with Next.js', -} diff --git a/website/src/pages/en/subgraphs/guides/grafting.mdx b/website/src/pages/en/subgraphs/guides/grafting.mdx deleted file mode 100644 index d9abe0e70d2a..000000000000 --- a/website/src/pages/en/subgraphs/guides/grafting.mdx +++ /dev/null @@ -1,202 +0,0 @@ ---- -title: Replace a Contract and Keep its History With Grafting ---- - -In this guide, you will learn how to build and deploy new Subgraphs by grafting existing Subgraphs. - -## What is Grafting? - -Grafting reuses the data from an existing Subgraph and starts indexing it at a later block. This is useful during development to get past simple errors in the mappings quickly or to temporarily get an existing Subgraph working again after it has failed. Also, it can be used when adding a feature to a Subgraph that takes long to index from scratch. - -The grafted Subgraph can use a GraphQL schema that is not identical to the one of the base Subgraph, but merely compatible with it. It has to be a valid Subgraph schema in its own right, but may deviate from the base Subgraph's schema in the following ways: - -- It adds or removes entity types -- It removes attributes from entity types -- It adds nullable attributes to entity types -- It turns non-nullable attributes into nullable attributes -- It adds values to enums -- It adds or removes interfaces -- It changes for which entity types an interface is implemented - -For more information, you can check: - -- [Grafting](/developing/creating-a-subgraph/#grafting-onto-existing-subgraphs) - -In this tutorial, we will be covering a basic use case. We will replace an existing contract with an identical contract (with a new address, but the same code). Then, graft the existing Subgraph onto the "base" Subgraph that tracks the new contract. - -## Important Note on Grafting When Upgrading to the Network - -> **Caution**: It is recommended to not use grafting for Subgraphs published to The Graph Network - -### Why Is This Important? - -Grafting is a powerful feature that allows you to "graft" one Subgraph onto another, effectively transferring historical data from the existing Subgraph to a new version. It is not possible to graft a Subgraph from The Graph Network back to Subgraph Studio. - -### Best Practices - -**Initial Migration**: when you first deploy your Subgraph to the decentralized network, do so without grafting. Ensure that the Subgraph is stable and functioning as expected. - -**Subsequent Updates**: once your Subgraph is live and stable on the decentralized network, you may use grafting for future versions to make the transition smoother and to preserve historical data. - -By adhering to these guidelines, you minimize risks and ensure a smoother migration process. - -## Building an Existing Subgraph - -Building Subgraphs is an essential part of The Graph, described more in depth [here](/subgraphs/quick-start/). To be able to build and deploy the existing Subgraph used in this tutorial, the following repo is provided: - -- [Subgraph example repo](https://github.com/Shiyasmohd/grafting-tutorial) - -> Note: The contract used in the Subgraph was taken from the following [Hackathon Starterkit](https://github.com/schmidsi/hackathon-starterkit). - -## Subgraph Manifest Definition - -The Subgraph manifest `subgraph.yaml` identifies the data sources for the Subgraph, the triggers of interest, and the functions that should be run in response to those triggers. See below for an example Subgraph manifest that you will use: - -```yaml -specVersion: 1.3.0 -schema: - file: ./schema.graphql -dataSources: - - kind: ethereum - name: Lock - network: sepolia - source: - address: '0xb3aabe721794b85fe4e72134795c2f93b4eb7e63' - abi: Lock - startBlock: 5955690 - mapping: - kind: ethereum/events - apiVersion: 0.0.9 - language: wasm/assemblyscript - entities: - - Withdrawal - abis: - - name: Lock - file: ./abis/Lock.json - eventHandlers: - - event: Withdrawal(uint256,uint256) - handler: handleWithdrawal - file: ./src/lock.ts -``` - -- The `Lock` data source is the abi and contract address we will get when we compile and deploy the contract -- The network should correspond to an indexed network being queried. Since we're running on Sepolia testnet, the network is `sepolia` -- The `mapping` section defines the triggers of interest and the functions that should be run in response to those triggers. In this case, we are listening for the `Withdrawal` event and calling the `handleWithdrawal` function when it is emitted. - -## Grafting Manifest Definition - -Grafting requires adding two new items to the original Subgraph manifest: - -```yaml ---- -features: - - grafting # feature name -graft: - base: Qm... # Subgraph ID of base Subgraph - block: 5956000 # block number -``` - -- `features:` is a list of all used [feature names](/developing/creating-a-subgraph/#experimental-features). -- `graft:` is a map of the `base` Subgraph and the block to graft on to. The `block` is the block number to start indexing from. The Graph will copy the data of the base Subgraph up to and including the given block and then continue indexing the new Subgraph from that block on. - -The `base` and `block` values can be found by deploying two Subgraphs: one for the base indexing and one with grafting - -## Deploying the Base Subgraph - -1. Go to [Subgraph Studio](https://thegraph.com/studio/) and create a Subgraph on Sepolia testnet called `graft-example` -2. Follow the directions in the `AUTH & DEPLOY` section on your Subgraph page in the `graft-example` folder from the repo -3. Once finished, verify the Subgraph is indexing properly. If you run the following command in The Graph Playground - -```graphql -{ - withdrawals(first: 5) { - id - amount - when - } -} -``` - -It returns something like this: - -``` -{ - "data": { - "withdrawals": [ - { - "id": "0xe8323d21c4f104607b10b0fff9fc24b9612b9488795dea8196b2d5f980d3dc1d0a000000", - "amount": "0", - "when": "1716394824" - }, - { - "id": "0xea1cee35036f2cacb72f2a336be3e54ab911f5bebd58f23400ebb8ecc5cfc45203000000", - "amount": "0", - "when": "1716394848" - } - ] - } -} -``` - -Once you have verified the Subgraph is indexing properly, you can quickly update the Subgraph with grafting. - -## Deploying the Grafting Subgraph - -The graft replacement subgraph.yaml will have a new contract address. This could happen when you update your dapp, redeploy a contract, etc. - -1. Go to [Subgraph Studio](https://thegraph.com/studio/) and create a Subgraph on Sepolia testnet called `graft-replacement` -2. Create a new manifest. The `subgraph.yaml` for `graph-replacement` contains a different contract address and new information about how it should graft. These are the `block` of the [last event emitted](https://sepolia.etherscan.io/tx/0xea1cee35036f2cacb72f2a336be3e54ab911f5bebd58f23400ebb8ecc5cfc452) you care about by the old contract and the `base` of the old Subgraph. The `base` Subgraph ID is the `Deployment ID` of your original `graph-example` Subgraph. You can find this in Subgraph Studio. -3. Follow the directions in the `AUTH & DEPLOY` section on your Subgraph page in the `graft-replacement` folder from the repo -4. Once finished, verify the Subgraph is indexing properly. If you run the following command in The Graph Playground - -```graphql -{ - withdrawals(first: 5) { - id - amount - when - } -} -``` - -It should return the following: - -``` -{ - "data": { - "withdrawals": [ - { - "id": "0xe8323d21c4f104607b10b0fff9fc24b9612b9488795dea8196b2d5f980d3dc1d0a000000", - "amount": "0", - "when": "1716394824" - }, - { - "id": "0xea1cee35036f2cacb72f2a336be3e54ab911f5bebd58f23400ebb8ecc5cfc45203000000", - "amount": "0", - "when": "1716394848" - }, - { - "id": "0x2410475f76a44754bae66d293d14eac34f98ec03a3689cbbb56a716d20b209af06000000", - "amount": "0", - "when": "1716429732" - } - ] - } -} -``` - -You can see that the `graft-replacement` Subgraph is indexing from older `graph-example` data and newer data from the new contract address. The original contract emitted two `Withdrawal` events, [Event 1](https://sepolia.etherscan.io/tx/0xe8323d21c4f104607b10b0fff9fc24b9612b9488795dea8196b2d5f980d3dc1d) and [Event 2](https://sepolia.etherscan.io/tx/0xea1cee35036f2cacb72f2a336be3e54ab911f5bebd58f23400ebb8ecc5cfc452). The new contract emitted one `Withdrawal` after, [Event 3](https://sepolia.etherscan.io/tx/0x2410475f76a44754bae66d293d14eac34f98ec03a3689cbbb56a716d20b209af). The two previously indexed transactions (Event 1 and 2) and the new transaction (Event 3) were combined together in the `graft-replacement` Subgraph. - -Congrats! You have successfully grafted a Subgraph onto another Subgraph. - -## Additional Resources - -If you want more experience with grafting, here are a few examples for popular contracts: - -- [Curve](https://github.com/messari/subgraphs/blob/master/subgraphs/curve-finance/protocols/curve-finance/config/templates/curve.template.yaml) -- [ERC-721](https://github.com/messari/subgraphs/blob/master/subgraphs/erc721-metadata/subgraph.yaml) -- [Uniswap](https://github.com/messari/subgraphs/blob/master/subgraphs/uniswap-v3-forks/protocols/uniswap-v3/config/templates/uniswapV3Template.yaml), - -To become even more of a Graph expert, consider learning about other ways to handle changes in underlying datasources. Alternatives like [Data Source Templates](/developing/creating-a-subgraph/#data-source-templates) can achieve similar results - -> Note: A lot of material from this article was taken from the previously published [Arweave article](/subgraphs/cookbook/arweave/) diff --git a/website/src/pages/en/subgraphs/guides/subgraph-debug-forking.mdx b/website/src/pages/en/subgraphs/guides/subgraph-debug-forking.mdx deleted file mode 100644 index 91aa7484d2ec..000000000000 --- a/website/src/pages/en/subgraphs/guides/subgraph-debug-forking.mdx +++ /dev/null @@ -1,101 +0,0 @@ ---- -title: Quick and Easy Subgraph Debugging Using Forks ---- - -As with many systems processing large amounts of data, The Graph's Indexers (Graph Nodes) may take quite some time to sync-up your Subgraph with the target blockchain. The discrepancy between quick changes with the purpose of debugging and long wait times needed for indexing is extremely counterproductive and we are well aware of that. This is why we are introducing **Subgraph forking**, developed by [LimeChain](https://limechain.tech/), and in this article I will show you how this feature can be used to substantially speed-up Subgraph debugging! - -## Ok, what is it? - -**Subgraph forking** is the process of lazily fetching entities from _another_ Subgraph's store (usually a remote one). - -In the context of debugging, **Subgraph forking** allows you to debug your failed Subgraph at block _X_ without needing to wait to sync-up to block _X_. - -## What?! How? - -When you deploy a Subgraph to a remote Graph Node for indexing and it fails at block _X_, the good news is that the Graph Node will still serve GraphQL queries using its store, which is synced-up to block _X_. That's great! This means we can take advantage of this "up-to-date" store to fix the bugs arising when indexing block _X_. - -In a nutshell, we are going to _fork the failing Subgraph_ from a remote Graph Node that is guaranteed to have the Subgraph indexed up to block _X_ in order to provide the locally deployed Subgraph being debugged at block _X_ an up-to-date view of the indexing state. - -## Please, show me some code! - -To stay focused on Subgraph debugging, let's keep things simple and run along with the [example-Subgraph](https://github.com/graphprotocol/graph-tooling/tree/main/examples/ethereum-gravatar) indexing the Ethereum Gravity smart contract. - -Here are the handlers defined for indexing `Gravatar`s, with no bugs whatsoever: - -```tsx -export function handleNewGravatar(event: NewGravatar): void { - let gravatar = new Gravatar(event.params.id.toHex().toString()) - gravatar.owner = event.params.owner - gravatar.displayName = event.params.displayName - gravatar.imageUrl = event.params.imageUrl - gravatar.save() -} - -export function handleUpdatedGravatar(event: UpdatedGravatar): void { - let gravatar = Gravatar.load(event.params.id.toI32().toString()) - if (gravatar == null) { - log.critical('Gravatar not found!', []) - return - } - gravatar.owner = event.params.owner - gravatar.displayName = event.params.displayName - gravatar.imageUrl = event.params.imageUrl - gravatar.save() -} -``` - -Oops, how unfortunate, when I deploy my perfect looking Subgraph to [Subgraph Studio](https://thegraph.com/studio/) it fails with the _"Gravatar not found!"_ error. - -The usual way to attempt a fix is: - -1. Make a change in the mappings source, which you believe will solve the issue (while I know it won't). -2. Re-deploy the Subgraph to [Subgraph Studio](https://thegraph.com/studio/) (or another remote Graph Node). -3. Wait for it to sync-up. -4. If it breaks again go back to 1, otherwise: Hooray! - -It is indeed pretty familiar to an ordinary debug process, but there is one step that horribly slows down the process: _3. Wait for it to sync-up._ - -Using **Subgraph forking** we can essentially eliminate this step. Here is how it looks: - -0. Spin-up a local Graph Node with the **_appropriate fork-base_** set. -1. Make a change in the mappings source, which you believe will solve the issue. -2. Deploy to the local Graph Node, **_forking the failing Subgraph_** and **_starting from the problematic block_**. -3. If it breaks again, go back to 1, otherwise: Hooray! - -Now, you may have 2 questions: - -1. fork-base what??? -2. Forking who?! - -And I answer: - -1. `fork-base` is the "base" URL, such that when the _subgraph id_ is appended the resulting URL (`/`) is a valid GraphQL endpoint for the Subgraph's store. -2. Forking is easy, no need to sweat: - -```bash -$ graph deploy --debug-fork --ipfs http://localhost:5001 --node http://localhost:8020 -``` - -Also, don't forget to set the `dataSources.source.startBlock` field in the Subgraph manifest to the number of the problematic block, so you can skip indexing unnecessary blocks and take advantage of the fork! - -So, here is what I do: - -1. I spin-up a local Graph Node ([here is how to do it](https://github.com/graphprotocol/graph-node#running-a-local-graph-node)) with the `fork-base` option set to: `https://api.thegraph.com/subgraphs/id/`, since I will fork a Subgraph, the buggy one I deployed earlier, from [Subgraph Studio](https://thegraph.com/studio/). - -``` -$ cargo run -p graph-node --release -- \ - --postgres-url postgresql://USERNAME[:PASSWORD]@localhost:5432/graph-node \ - --ethereum-rpc NETWORK_NAME:[CAPABILITIES]:URL \ - --ipfs 127.0.0.1:5001 - --fork-base https://api.thegraph.com/subgraphs/id/ -``` - -2. After careful inspection I notice that there is a mismatch in the `id` representations used when indexing `Gravatar`s in my two handlers. While `handleNewGravatar` converts it to a hex (`event.params.id.toHex()`), `handleUpdatedGravatar` uses an int32 (`event.params.id.toI32()`) which causes the `handleUpdatedGravatar` to panic with "Gravatar not found!". I make them both convert the `id` to a hex. -3. After I made the changes I deploy my Subgraph to the local Graph Node, **_forking the failing Subgraph_** and setting `dataSources.source.startBlock` to `6190343` in `subgraph.yaml`: - -```bash -$ graph deploy gravity --debug-fork QmNp169tKvomnH3cPXTfGg4ZEhAHA6kEq5oy1XDqAxqHmW --ipfs http://localhost:5001 --node http://localhost:8020 -``` - -4. I inspect the logs produced by the local Graph Node and, Hooray!, everything seems to be working. -5. I deploy my now bug-free Subgraph to a remote Graph Node and live happily ever after! (no potatoes tho) diff --git a/website/src/pages/en/subgraphs/tooling/skills.mdx b/website/src/pages/en/subgraphs/tooling/skills.mdx index e87c494aead2..f06165a15599 100644 --- a/website/src/pages/en/subgraphs/tooling/skills.mdx +++ b/website/src/pages/en/subgraphs/tooling/skills.mdx @@ -94,8 +94,8 @@ Once installed, the AI assistant will have access to subgraph development expert ## Resources -- [Subgraph Best Practices](/subgraphs/best-practices/pruning/) -- [Subgraph Composition](/subgraphs/guides/subgraph-composition/) - Combine multiple subgraphs +- [Subgraph Guides & Best Practices](/subgraphs/best-practices/overview/) +- [Subgraph Composition](/subgraphs/best-practices/subgraph-composition/) - Combine multiple subgraphs - [Subgraph Linter](/subgraphs/developing/subgraph-linter/) - Static analysis tool - [Subgraph Uncrashable](/subgraphs/developing/subgraph-uncrashable/) - Safe code generation - [Matchstick Testing Framework](/subgraphs/tooling/unit-testing-framework/) diff --git a/website/src/pages/en/substreams/overview.mdx b/website/src/pages/en/substreams/overview.mdx index 7e57d0b5a491..41845e23e44f 100644 --- a/website/src/pages/en/substreams/overview.mdx +++ b/website/src/pages/en/substreams/overview.mdx @@ -14,7 +14,7 @@ Substreams is a powerful parallel blockchain indexing technology designed to enh ## Substreams Benefits - **Accelerated Indexing**: Boost Subgraph indexing time with a parallelized engine for quicker data retrieval and processing. -- **Multi-Chain Support**: Expand indexing capabilities beyond EVM-based chains, supporting ecosystems like Solana, Injective, Starknet, and Vara. +- **Multi-Chain Support**: Expand indexing capabilities beyond EVM-based chains, supporting ecosystems like Solana, Bitcoin, Injective, and Stellar. - **Enhanced Data Model**: Access comprehensive data, including the `trace` level data on EVM or account changes on Solana, while efficiently managing forks/disconnections. - **Multi-Sink Support:** For Subgraph, Postgres database, Clickhouse, and Mongo database. diff --git a/website/src/pages/en/substreams/quick-start.mdx b/website/src/pages/en/substreams/quick-start.mdx index c9f310b847a1..cf2f3f0f7caa 100644 --- a/website/src/pages/en/substreams/quick-start.mdx +++ b/website/src/pages/en/substreams/quick-start.mdx @@ -29,9 +29,7 @@ If you can't find a Substreams package that meets your specific needs, you can d - [EVM](https://docs.substreams.dev/tutorials/intro-to-tutorials/evm) - [Solana](https://docs.substreams.dev/tutorials/intro-to-tutorials/on-solana) - [TRON](https://docs.substreams.dev/tutorials/intro-to-tutorials/tron) -- [Starknet](https://docs.substreams.dev/tutorials/intro-to-tutorials/starknet) - [Injective](https://docs.substreams.dev/tutorials/intro-to-tutorials/on-cosmos/injective) -- [MANTRA](https://docs.substreams.dev/tutorials/intro-to-tutorials/on-cosmos/mantra) - [Stellar](https://docs.substreams.dev/tutorials/intro-to-tutorials/stellar) To build and optimize your Substreams from zero, use the minimal path within the [Dev Container](/substreams/developing/dev-container/). diff --git a/website/src/supportedNetworks/customContent/README.md b/website/src/supportedNetworks/customContent/README.md index d433f914506f..163dbee72825 100644 --- a/website/src/supportedNetworks/customContent/README.md +++ b/website/src/supportedNetworks/customContent/README.md @@ -41,7 +41,7 @@ Common, repeated sections live in shared MDX partials (prefixed `_`) so a single Two partials cover the standard "Indexing _{Chain}_ with Substreams" section for EVM networks, differing only by the block model the network is served with: - `_substreams-extended.mdx` — **extended** EVM block model (full transaction, call, and event data). Used by chains like BSC, Polygon, Monad, Ink. -- `_substreams-base.mdx` — **base** EVM block model (block, transaction, and event/log data). Used by chains like Blast, MegaETH, TRON EVM. +- `_substreams-base.mdx` — **base** EVM block model (block, transaction, and event/log data). Used by chains like MegaETH, TRON EVM, zkSync Era. ```mdx import SubstreamsExtended from './_substreams-extended.mdx' diff --git a/website/src/supportedNetworks/customContent/base.mdx b/website/src/supportedNetworks/customContent/base.mdx index f734fec76ce5..901aec177ec1 100644 --- a/website/src/supportedNetworks/customContent/base.mdx +++ b/website/src/supportedNetworks/customContent/base.mdx @@ -1,15 +1,67 @@ import SubstreamsExtended from './_substreams-extended.mdx' +import { Card } from '@/components' import SubgraphCommunity from './_subgraph-community.mdx' +import { Subgraph } from '@edgeandnode/gds/icons' +import { ExploreSectionHeading } from '@/supportedNetworks/GetApiKeyButton' ## Indexing Data on Base -Base is an Ethereum Layer 2 built on the OP Stack and part of the Optimism Superchain, incubated by Coinbase, with one of the fastest-growing DeFi, consumer, and onchain-social ecosystems in crypto. The Graph can index that onchain activity, including DEX swaps, pools, and liquidity; lending and borrowing positions; token transfers and holder balances; and contract events. That powers DeFi frontends, analytics dashboards, block explorers, portfolio trackers, and onchain agents without teams having to run their own indexing infrastructure. +Base is an Ethereum Layer 2 built on the OP Stack and part of the Optimism Superchain. Incubated by Coinbase and live since August 2023, it offers low fees and fast confirmations while settling to Ethereum. It hosts one of the fastest-growing DeFi, consumer, and onchain-social ecosystems in crypto, with protocols like Aerodrome and Uniswap (DEXs), Morpho, Aave, and Moonwell (lending), and thousands of consumer apps. + +The Graph can index that onchain activity, including DEX swaps, pools, and liquidity; lending and borrowing positions; token transfers and holder balances; and contract events. That powers DeFi frontends, analytics dashboards, block explorers, portfolio trackers, and onchain agents without teams having to run their own indexing infrastructure. Base is supported by The Graph in two ways: - **[Subgraphs](/subgraphs/developing/deploying-publishing/publishing-a-subgraph/)**: The blockchain indexing standard perfect for small to medium-sized projects that need hosted APIs. Data consumers manage their service in [Subgraph Studio](/subgraphs/providers/subgraph-studio/introduction/) while Indexers on The Graph Network sync and serve queries. - **[Substreams](/substreams/quick-start/)**: A real-time streaming engine for transaction-heavy projects, high-throughput chains, and institutions that need custom data infrastructure. Data consumers can manage their service and get indexing via [The Graph Market](https://thegraph.market/). +--- + + + +Before you build, check whether the data you need is already live. Many of the most-used protocols on Base publish Subgraphs on The Graph that you can query today: + +
+ } + href="https://thegraph.com/explorer/subgraphs/HNCFA9TyBqpo5qpe6QreQABAA1kV8g46mhkCcicu6v2R" + /> + } + href="https://thegraph.com/explorer/subgraphs/43Hwfi3dJSoGpyas9VwNoDAv55yjgGrPpNSmbQZArzMG" + /> + } + href="https://thegraph.com/explorer/subgraphs/GENunSHWLBXm59mBSgPzQ8metBEp9YDfdqwFr91Av1UM" + /> + } + href="https://thegraph.com/explorer/subgraphs/71ZTy1veF9twER9CLMnPWeLQ7GZcwKsjmygejrgKirqs" + /> + } + href="https://thegraph.com/explorer/subgraphs/D7mapexM5ZsQckLJai2FawTKXJ7CqYGKM8PErnS3cJi9" + /> + } + href="https://thegraph.com/explorer/subgraphs/33ex1ExmYQtwGVwri1AP3oMFPGSce6YbocBP7fWbsBrg" + /> +
+ +Browse [Graph Explorer](https://thegraph.com/explorer/) to search all published Subgraphs on Base by protocol, signal, and query volume before deciding to build your own. + diff --git a/website/src/supportedNetworks/customContent/blast-mainnet.mdx b/website/src/supportedNetworks/customContent/blast-mainnet.mdx deleted file mode 100644 index 3115883753b0..000000000000 --- a/website/src/supportedNetworks/customContent/blast-mainnet.mdx +++ /dev/null @@ -1,9 +0,0 @@ -import SubstreamsBase from './_substreams-base.mdx' - -## Indexing Data on Blast - -Blast is an Ethereum Layer 2 built on the OP Stack that offers native yield on ETH and stablecoins for its DeFi and dApp ecosystem. The Graph can stream that onchain activity — DEX swaps, pools, and liquidity; lending and borrowing positions; token transfers and holder balances; and contract events — as real-time data with Substreams. - -Blast is supported by The Graph with Substreams — a parallelized, real-time streaming engine for high-throughput use cases — via [Pinax Network](https://app.pinax.network/docs?from=nav). - - diff --git a/website/src/supportedNetworks/customContent/elysium.mdx b/website/src/supportedNetworks/customContent/elysium.mdx new file mode 100644 index 000000000000..4a3659a3c9e1 --- /dev/null +++ b/website/src/supportedNetworks/customContent/elysium.mdx @@ -0,0 +1,9 @@ +import SubstreamsExtended from './_substreams-extended.mdx' + +## Indexing Data on Elysium + +Elysium is a high-throughput Arbitrum Orbit Layer 2 built by Kinetiq for the Hyperliquid ecosystem. It settles to HyperEVM, uses HYPE as its gas token, and targets 100–200 ms blocks for spot trading, market making, token launches, and DeFi apps. The Graph can stream that onchain activity — DEX swaps, pools, and liquidity; lending and borrowing positions; token transfers and holder balances; and contract events — as real-time data with Substreams. + +Elysium is supported by The Graph with Substreams — a parallelized, real-time streaming engine for high-throughput use cases — via [The Graph Market](https://thegraph.market/). + + diff --git a/website/src/supportedNetworks/customContent/index.ts b/website/src/supportedNetworks/customContent/index.ts index 1d49ead72a0f..f18791a5da41 100644 --- a/website/src/supportedNetworks/customContent/index.ts +++ b/website/src/supportedNetworks/customContent/index.ts @@ -8,7 +8,6 @@ import ArcTestnet from './arc-testnet.mdx' import Avalanche from './avalanche.mdx' import Base from './base.mdx' import BaseSepolia from './base-sepolia.mdx' -import Blast from './blast-mainnet.mdx' import Boba from './boba.mdx' import BobaTestnet from './boba-testnet.mdx' import Bsc from './bsc.mdx' @@ -18,6 +17,7 @@ import CeloSepolia from './celo-sepolia.mdx' import Chapel from './chapel.mdx' import Chiliz from './chiliz.mdx' import ChilizTestnet from './chiliz-testnet.mdx' +import Elysium from './elysium.mdx' import Vaulta from './eos.mdx' import Etherlink from './etherlink-mainnet.mdx' import EtherlinkShadownet from './etherlink-shadownet.mdx' @@ -54,7 +54,6 @@ import OptimismSepolia from './optimism-sepolia.mdx' import Peaq from './peaq.mdx' import PolygonAmoy from './polygon-amoy.mdx' import Robinhood from './robinhood.mdx' -import RobinhoodSepolia from './robinhood-sepolia.mdx' import Rootstock from './rootstock.mdx' import RootstockTestnet from './rootstock-testnet.mdx' import Scroll from './scroll.mdx' @@ -71,8 +70,6 @@ import SoneiumTestnet from './soneium-testnet.mdx' import Sonic from './sonic.mdx' import SonicTestnet from './sonic-testnet.mdx' import Stable from './stable.mdx' -import Starknet from './starknet-mainnet.mdx' -import StarknetTestnet from './starknet-testnet.mdx' import Stellar from './stellar.mdx' import StellarTestnet from './stellar-testnet.mdx' import Tempo from './tempo.mdx' @@ -107,16 +104,17 @@ import Zora from './zora.mdx' */ export const customNetworkContent: Record = { anubis: Anubis, + base: Base, bsc: Bsc, matic: Polygon, ink: Ink, 'hyper-evm': HyperEvm, + elysium: Elysium, robinhood: Robinhood, monad: Monad, worldchain: WorldChain, zora: Zora, btc: Btc, - 'blast-mainnet': Blast, 'mainnet-cl': EthereumBeacon, 'hoodi-cl': HoodiBeacon, 'sepolia-cl': SepoliaBeacon, @@ -127,7 +125,6 @@ export const customNetworkContent: Record = { 'solana-devnet': SolanaDevnet, 'solana-mainnet-beta': Solana, 'solana-accounts': SolanaAccounts, - 'starknet-mainnet': Starknet, stellar: Stellar, 'tron-evm': TronEvm, tron: Tron, @@ -136,7 +133,6 @@ export const customNetworkContent: Record = { // Substreams-only custom content (subgraph sections to be added later) 'arbitrum-one': ArbitrumOne, arc: Arc, - base: Base, 'injective-evm': InjectiveEvm, linea: Linea, mainnet: Ethereum, @@ -151,7 +147,6 @@ export const customNetworkContent: Record = { 'linea-sepolia': LineaSepolia, 'optimism-sepolia': OptimismSepolia, 'polygon-amoy': PolygonAmoy, - 'robinhood-sepolia': RobinhoodSepolia, sepolia: Sepolia, 'soneium-testnet': SoneiumTestnet, 'unichain-testnet': UnichainTestnet, @@ -159,7 +154,6 @@ export const customNetworkContent: Record = { celo: Celo, tempo: Tempo, 'xlayer-mainnet': XLayer, - 'starknet-testnet': StarknetTestnet, 'stellar-testnet': StellarTestnet, 'injective-testnet': InjectiveTestnet, jungle4: Jungle4, diff --git a/website/src/supportedNetworks/customContent/robinhood-sepolia.mdx b/website/src/supportedNetworks/customContent/robinhood-sepolia.mdx deleted file mode 100644 index e96c0080275c..000000000000 --- a/website/src/supportedNetworks/customContent/robinhood-sepolia.mdx +++ /dev/null @@ -1,9 +0,0 @@ -import SubstreamsExtended from './_substreams-extended.mdx' - -## Indexing Data on Robinhood Chain Testnet - -Robinhood Chain Testnet is the public test network for Robinhood Chain, an Arbitrum-based Ethereum Layer 2 for tokenized real-world assets, where teams deploy and validate contracts before mainnet. The Graph can stream that onchain activity — DEX swaps, pools, and liquidity; lending and borrowing positions; token transfers and holder balances; and contract events — as real-time data with Substreams. - -Robinhood Chain Testnet is supported by The Graph with Substreams — a parallelized, real-time streaming engine for high-throughput use cases — via [Pinax Network](https://app.pinax.network/docs?from=nav). - - diff --git a/website/src/supportedNetworks/customContent/starknet-mainnet.mdx b/website/src/supportedNetworks/customContent/starknet-mainnet.mdx deleted file mode 100644 index 3291a8a525cc..000000000000 --- a/website/src/supportedNetworks/customContent/starknet-mainnet.mdx +++ /dev/null @@ -1,30 +0,0 @@ -import { ProductSectionHeading } from '@/supportedNetworks/GetApiKeyButton' - -## Indexing Data on Starknet - -Starknet is an Ethereum Layer 2 validity rollup powered by STARK proofs, using the Cairo programming model for scalable, low-cost execution. The Graph can stream that onchain activity — transactions, DEX swaps, token transfers and balances, and contract events — as real-time data with Substreams. - -Starknet is supported by The Graph with Substreams — a parallelized, real-time streaming engine for high-throughput use cases — via [The Graph Market](https://thegraph.market/). - ---- - - - -[Substreams](/substreams/overview/) is a parallelized, real-time streaming engine for extracting and transforming blockchain data. Where a Subgraph indexes into a hosted GraphQL API, Substreams lets you consume high-throughput data directly or via [10+ supported sinks](/substreams/developing/sinks/). Substreams streams Starknet's blocks, transactions, and events. Modules are written in Rust and composed into reusable packages. - -### Find an Existing Module on Substreams.dev - -Before writing any code, check the [Substreams Registry](https://substreams.dev/) for a package that already does what you need. The registry hosts community- and team-built modules covering common onchain data that you can run as-is or compose into your own pipeline. Search for the protocol or data you want on Starknet; if a module fits, you can consume its output directly or import it as a dependency in your own project. - -### Develop Your Own Substreams Module - -If no existing module fits, you can build one. The [Substreams Quick Start](/substreams/overview/) is the full walkthrough. In short: - -1. Install the Substreams CLI and get a key at [The Graph Market](https://thegraph.market/). -2. Scaffold a project with `substreams init` and follow the prompts for Starknet. -3. Write your extraction and transformation logic in Rust, then run `substreams build`. -4. Stream the output with `substreams gui` (or `substreams run`) to iterate, then publish your package to the [Substreams Registry](https://substreams.dev/) to reuse or share it. diff --git a/website/src/supportedNetworks/customContent/starknet-testnet.mdx b/website/src/supportedNetworks/customContent/starknet-testnet.mdx deleted file mode 100644 index 1d8ae2680253..000000000000 --- a/website/src/supportedNetworks/customContent/starknet-testnet.mdx +++ /dev/null @@ -1,30 +0,0 @@ -import { ProductSectionHeading } from '@/supportedNetworks/GetApiKeyButton' - -## Indexing Data on Starknet Sepolia Testnet - -Starknet Sepolia Testnet is the public test network for Starknet, an Ethereum Layer 2 validity rollup powered by STARK proofs and the Cairo programming model, where teams deploy and validate contracts before mainnet. The Graph can stream that onchain activity — transactions, DEX swaps, token transfers and balances, and contract events — as real-time data with Substreams. - -Starknet Sepolia Testnet is supported by The Graph with Substreams — a parallelized, real-time streaming engine for high-throughput use cases — via [The Graph Market](https://thegraph.market/). - ---- - - - -[Substreams](/substreams/overview/) is a parallelized, real-time streaming engine for extracting and transforming blockchain data. Where a Subgraph indexes into a hosted GraphQL API, Substreams lets you consume high-throughput data directly or via [10+ supported sinks](/substreams/developing/sinks/). Substreams streams Starknet Sepolia Testnet's blocks, transactions, and events. Modules are written in Rust and composed into reusable packages. - -### Find an Existing Module on Substreams.dev - -Before writing any code, check the [Substreams Registry](https://substreams.dev/) for a package that already does what you need. The registry hosts community- and team-built modules covering common onchain data that you can run as-is or compose into your own pipeline. Search for the protocol or data you want on Starknet Sepolia Testnet; if a module fits, you can consume its output directly or import it as a dependency in your own project. - -### Develop Your Own Substreams Module - -If no existing module fits, you can build one. The [Substreams Quick Start](/substreams/overview/) is the full walkthrough. In short: - -1. Install the Substreams CLI and get a key at [The Graph Market](https://thegraph.market/). -2. Scaffold a project with `substreams init` and follow the prompts for Starknet. -3. Write your extraction and transformation logic in Rust, then run `substreams build`. -4. Stream the output with `substreams gui` (or `substreams run`) to iterate, then publish your package to the [Substreams Registry](https://substreams.dev/) to reuse or share it. diff --git a/website/src/supportedNetworks/customContent/zksync-era.mdx b/website/src/supportedNetworks/customContent/zksync-era.mdx index b69df78359a7..8940f78defd5 100644 --- a/website/src/supportedNetworks/customContent/zksync-era.mdx +++ b/website/src/supportedNetworks/customContent/zksync-era.mdx @@ -1,10 +1,14 @@ +import SubstreamsBase from './_substreams-base.mdx' import SubgraphCommunity from './_subgraph-community.mdx' ## Indexing Data on zkSync Era -zkSync Era is an Ethereum Layer 2 zkEVM rollup built by Matter Labs, using zero-knowledge proofs for scalable, low-cost execution with full EVM compatibility. The Graph can index that onchain activity — DEX swaps, pools, and liquidity; lending and borrowing positions; token transfers and holder balances; and contract events — and serve it over GraphQL with Subgraphs. +zkSync Era is an Ethereum Layer 2 zkEVM rollup built by Matter Labs, using zero-knowledge proofs for scalable, low-cost execution with full EVM compatibility. The Graph can index that onchain activity — DEX swaps, pools, and liquidity; lending and borrowing positions; token transfers and holder balances; and contract events — and serve it over GraphQL with Subgraphs or stream it in real time with Substreams. -zkSync Era is supported by The Graph with Subgraphs — open APIs you publish to The Graph Network, where Indexers sync and serve the data while you manage your service in [Subgraph Studio](/subgraphs/providers/subgraph-studio/introduction/). +zkSync Era is supported by The Graph in two ways: + +- **[Subgraphs](/subgraphs/developing/deploying-publishing/publishing-a-subgraph/)**: The blockchain indexing standard perfect for small to medium-sized projects that need hosted APIs. Data consumers manage their service in [Subgraph Studio](/subgraphs/providers/subgraph-studio/introduction/) while Indexers on The Graph Network sync and serve queries. +- **[Substreams](/substreams/quick-start/)**: A real-time streaming engine for transaction-heavy projects, high-throughput chains, and institutions that need custom data infrastructure. Data consumers can manage their service and get indexing via [Pinax Network](https://app.pinax.network/docs?from=nav). + + diff --git a/website/src/supportedNetworks/slugs.ts b/website/src/supportedNetworks/slugs.ts index 68e57c810cf3..1c935418b082 100644 --- a/website/src/supportedNetworks/slugs.ts +++ b/website/src/supportedNetworks/slugs.ts @@ -5,7 +5,6 @@ // whose page and metadata render for that slug. export const networkSlugs: Record = { btc: 'bitcoin', - 'blast-mainnet': 'blast', 'mainnet-cl': 'ethereum-beacon', 'hoodi-cl': 'ethereum-hoodi', 'sepolia-cl': 'ethereum-sepolia', From b887141d981cd2ab08e3d636ff0d92c894c22169 Mon Sep 17 00:00:00 2001 From: Brandon Kramer Date: Mon, 16 Nov 2026 19:08:12 +0000 Subject: [PATCH 2/4] Fix first heading level on Next.js API key and composition pages Promote the intro headings to h2 so each page's first heading is rank 2 (remark-lint-first-heading-level). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HpT75uYTePhuL7KFioUPqS --- .../en/subgraphs/best-practices/secure-api-keys-nextjs.mdx | 6 +++--- .../en/subgraphs/best-practices/subgraph-composition.mdx | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/website/src/pages/en/subgraphs/best-practices/secure-api-keys-nextjs.mdx b/website/src/pages/en/subgraphs/best-practices/secure-api-keys-nextjs.mdx index aca7fa9192ab..d69e582d329a 100644 --- a/website/src/pages/en/subgraphs/best-practices/secure-api-keys-nextjs.mdx +++ b/website/src/pages/en/subgraphs/best-practices/secure-api-keys-nextjs.mdx @@ -6,13 +6,13 @@ An API key in frontend code is visible to anyone who opens the browser's develop This guide shows how to create a Next.js server component that queries a Subgraph while keeping the API key out of the frontend. -### Caveats +## Caveats - Next.js server components do not protect API keys from being drained using denial of service attacks. - The Graph Network gateways have denial of service detection and mitigation strategies in place, however using server components may weaken these protections. - Next.js server components introduce centralization risks as the server can go down. -### Why It's Needed +## Why It's Needed In a standard React application, API keys included in the frontend code can be exposed to the client-side, posing a security risk. While `.env` files are commonly used, they don't fully protect the keys since React's code is executed on the client side, exposing the API key in the headers. Next.js Server Components address this issue by handling sensitive operations server-side. @@ -20,7 +20,7 @@ In a standard React application, API keys included in the frontend code can be e ![Client-side rendering](/img/api-key-client-side-rendering.png) -### Prerequisites +## Prerequisites - An API key from [Subgraph Studio](https://thegraph.com/studio) - Basic knowledge of Next.js and React. diff --git a/website/src/pages/en/subgraphs/best-practices/subgraph-composition.mdx b/website/src/pages/en/subgraphs/best-practices/subgraph-composition.mdx index 201eed2e7d3f..a85a3dbecdf9 100644 --- a/website/src/pages/en/subgraphs/best-practices/subgraph-composition.mdx +++ b/website/src/pages/en/subgraphs/best-practices/subgraph-composition.mdx @@ -4,7 +4,7 @@ title: Combine Data with Subgraph Composition Subgraph composition lets you build a Subgraph on top of other Subgraphs. Instead of one large Subgraph that indexes everything, you maintain small, focused source Subgraphs, then combine their data into a single composed Subgraph for your app. Each source syncs and resyncs on its own, which speeds up development and recovery. -### Benefits of Composition +## Benefits of Composition Subgraph composition is a powerful feature for scaling, allowing you to: From 5e0adac574a62964448f9bb500b3b6648e7e3a20 Mon Sep 17 00:00:00 2001 From: Brandon Kramer Date: Tue, 17 Nov 2026 03:24:54 +0000 Subject: [PATCH 3/4] Address review feedback on Guides & Best Practices - Declared eth_call examples: use the manifest event signature format `TransferWithPool(indexed address,indexed address,uint256,indexed bytes32)` - Overview: nested sorting works one level deep (String/ID fields), so recommend storing frequent sort keys on the entity rather than saying nested sorting is impossible - Overview: loadInBlock returns entities created or updated in the current block; a non-null result does not prove the entity is new - Overview: gnd dev --watch only redeploys the build output; run gnd build --watch (or graph build --watch) alongside it to recompile - Grafting: switching to Bytes IDs is not graft-compatible, but mutable to immutable is supported when the copied history has no updates or deletions - Grafting: a non-grafted republish after a contract replacement needs data sources for both contracts, and a single-data-source graft stops indexing the old contract after the graft block Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HpT75uYTePhuL7KFioUPqS --- .../en/subgraphs/best-practices/avoid-eth-calls.mdx | 2 +- .../src/pages/en/subgraphs/best-practices/grafting.mdx | 5 ++++- .../src/pages/en/subgraphs/best-practices/overview.mdx | 10 +++++----- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/website/src/pages/en/subgraphs/best-practices/avoid-eth-calls.mdx b/website/src/pages/en/subgraphs/best-practices/avoid-eth-calls.mdx index a4e7fd8fbadf..94538326931b 100644 --- a/website/src/pages/en/subgraphs/best-practices/avoid-eth-calls.mdx +++ b/website/src/pages/en/subgraphs/best-practices/avoid-eth-calls.mdx @@ -90,7 +90,7 @@ For the `eth_calls` that can not be eliminated, the runtime overhead they introd You can declare `eth_calls` only for event handlers, and only with `specVersion` 1.2.0 or later. In the manifest, add a `calls` section to the handler: ```yaml -event: TransferWithPool(address indexed, address indexed, uint256, bytes32 indexed) +event: TransferWithPool(indexed address,indexed address,uint256,indexed bytes32) handler: handleTransferWithPool calls: ERC20.poolInfo: ERC20[event.address].getPoolInfo(event.params.to) diff --git a/website/src/pages/en/subgraphs/best-practices/grafting.mdx b/website/src/pages/en/subgraphs/best-practices/grafting.mdx index 59ff840736c4..126a7f312c2a 100644 --- a/website/src/pages/en/subgraphs/best-practices/grafting.mdx +++ b/website/src/pages/en/subgraphs/best-practices/grafting.mdx @@ -38,7 +38,7 @@ The grafted Subgraph's schema does not have to match the base schema exactly, bu - It adds or removes interfaces - It changes for which entity types an interface is implemented -Any other change, such as changing a field's type or making a nullable field non-nullable, is not graft-compatible. Schema optimizations such as switching to [immutable entities or `Bytes` IDs](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) also break compatibility. +Any other change, such as changing a field's type or making a nullable field non-nullable, is not graft-compatible. That includes switching an entity's `id` to [`Bytes`](/subgraphs/best-practices/immutable-entities-bytes-as-ids/). Marking a mutable entity as [immutable](/subgraphs/best-practices/immutable-entities-bytes-as-ids/) is supported only if the copied history contains no updates or deletions of that entity. ## When Not to Use Grafting @@ -231,9 +231,12 @@ You can see that the `graft-replacement` Subgraph is indexing from older `graft- The grafted Subgraph now serves the full history across both contracts. +> Note: The grafted manifest in this walkthrough has one data source, for the new contract. It copies the old contract's data up to the graft block, but it does not index any events the old contract emits after that block. If the old contract can still emit events you need, keep a data source for it in the grafted manifest too. + ## After Grafting - **Republish without grafting:** Once the grafted version is stable, deploy a non-grafted version for long-term maintenance. Relying on a graft indefinitely complicates future updates. +- **Keep both contracts after a contract replacement:** A non-grafted version indexes from scratch, so it needs two data sources, one for the old contract (with its original `startBlock`) and one for the new contract. If you only replace the address, the non-grafted version indexes only the new contract's data. - **Move your apps:** Point every app and service at the new, non-grafted version. - **Keep the old version serving** until the new one has fully synced. diff --git a/website/src/pages/en/subgraphs/best-practices/overview.mdx b/website/src/pages/en/subgraphs/best-practices/overview.mdx index 1ea4b6d54d52..ebcb0588afdc 100644 --- a/website/src/pages/en/subgraphs/best-practices/overview.mdx +++ b/website/src/pages/en/subgraphs/best-practices/overview.mdx @@ -194,7 +194,7 @@ type Stats @aggregation(intervals: ["hour", "day"], source: "Data") { ### Types and Query-Friendly Modeling - **Sort by numbers, not strings:** Store amounts as `Int` or `BigInt`. String order puts "10" before "2". -- **Put sort keys on the entity itself:** GraphQL queries cannot order by a nested field, so add a `createdAt` or similar field where you sort. +- **Put frequent sort keys on the entity itself:** Queries can sort by a field of a related entity, one level deep (for example `orderBy: owner__name`), but this is slower than sorting by the entity's own field and supports only `String` and `ID` fields. For sort keys your app uses often, such as `createdAt`, store the field on the entity. - **Store facts, compute state:** Do not store a value that changes with time but has no event, such as a "LOCKED" status after a lock period. Store the timestamps and durations; [compute the state in the app](https://thegraph.com/blog/subgraph-devex-tips/). - **Keep counters current:** Fields such as `positionCount` let apps paginate without counting. Update every affected entity in the same handler. - **Avoid deep nesting:** Give frequently read data a direct path, not a chain of five nested relationships. @@ -240,7 +240,7 @@ Requires `specVersion` 1.2.0. ```yaml eventHandlers: - - event: TransferWithPool(address indexed, address indexed, uint256, bytes32 indexed) + - event: TransferWithPool(indexed address,indexed address,uint256,indexed bytes32) handler: handleTransferWithPool calls: poolInfo: ERC20[event.address].getPoolInfo(event.params.to) @@ -254,7 +254,7 @@ eventHandlers: ### Entity Reads and Writes - **Save only what changed:** Do not load and save entities that the handler does not modify. -- **Use `loadInBlock()` for entities created earlier in the same block:** It skips the database round trip for a [lookup that would miss](/subgraphs/developing/creating/graph-ts/api/). It returns `null` if the entity was not created in the current block, even if it exists in the store. +- **Use `loadInBlock()` for entities written earlier in the same block:** It returns an entity that was created, or updated and saved, in the current block, and skips the database round trip for a [lookup that would miss](/subgraphs/developing/creating/graph-ts/api/#looking-up-entities-created-within-a-block). It returns `null` if the entity was not written in the current block, even if it exists in the store. A non-null result does not prove the entity was newly created. - **Do not overwrite fresh data with a stale copy:** If a helper loads and saves an entity, reload it in the caller before you save. [Subgraph Linter](/subgraphs/tooling/subgraph-linter/) flags this as `entity-overwrite`. - **Update every affected entity in one handler:** A stake event can change the protocol total, the user total, and the position. Update all three so that [totals stay in sync](https://thegraph.com/blog/subgraph-devex-tips/). - **Use `try_` for calls that can revert:** A reverting `decimals()` or `symbol()` is one of the most common causes of a crashed Subgraph. Set a default value and keep indexing. @@ -308,7 +308,7 @@ indexerHints: ### Iterate Faster During Development - **Use a recent `startBlock` locally:** Restore the real block before you publish ([Graph Node Dev Mode](/subgraphs/developing/creating/graph-node-dev/)). -- **Run Graph Node in watch mode:** `gnd dev --watch` rebuilds when files change. Always pass `--node` to `gnd deploy`; without it, `gnd deploy` targets Subgraph Studio. +- **Run two watchers for hot reloading:** `gnd dev --watch` watches the build output and redeploys it, but it does not compile. Run `gnd build --watch` (or `graph build --watch`) in a second terminal so that edits to mappings, the schema, or the manifest are recompiled; `gnd dev --watch` then picks up the new build. Always pass `--node` to `gnd deploy`; without it, `gnd deploy` targets Subgraph Studio. - **Debug with forking:** Point a local Graph Node at a remote deployment that failed at block X. The local node fetches entities from the remote store, so you [reproduce the failure without syncing to block X](/subgraphs/best-practices/subgraph-debug-forking/). - **Use operator staging for full-history checks:** Some Gateway Operators offer a staging environment with logs, for syncs too large to run locally. - **Expect faster re-syncs on hosted providers:** Some providers cache `eth_call` and log results, so a second sync of the same contracts reuses them ([Goldsky](https://docs.goldsky.com/subgraphs/guides/performance) documents a permanent RPC cache). @@ -332,7 +332,7 @@ graft: ``` - **Graft-safe:** changes inside mappings or the manifest, such as a fixed handler or a removed handler. -- **Not graft-safe:** changing or removing field types, and large mapping rewrites. Schema changes such as immutable entities or `Bytes` IDs also break graft compatibility. +- **Not graft-safe:** changing field types (including switching IDs to `Bytes`), making nullable fields non-nullable, and large mapping rewrites. Marking an entity immutable is supported only if the copied history has no updates or deletions of it. See [compatible schema changes](/subgraphs/best-practices/grafting/#compatible-schema-changes). - **Grafting does not make later blocks faster.** [Remaining blocks index faster only if the new code does less work per event](https://docs.goldsky.com/subgraphs/guides/performance). - **Not for The Graph Network.** The docs do not recommend grafting for mainnet Subgraphs on the network. Some Indexers may not support it. - **Pruning conflict:** you cannot graft at a pruned block. From eb7dd002fad9b891c512093dc19d3688b53cffd0 Mon Sep 17 00:00:00 2001 From: Brandon Kramer Date: Tue, 17 Nov 2026 20:06:48 +0000 Subject: [PATCH 4/4] Network Subgraph template: run a build watcher alongside gnd --watch gnd --watch only redeploys the build output and does not compile, so the local verification step now starts graph build --watch (or gnd build --watch) in one terminal and gnd --watch in a second terminal. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HpT75uYTePhuL7KFioUPqS --- .../customContent/_subgraph-community.mdx | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/website/src/supportedNetworks/customContent/_subgraph-community.mdx b/website/src/supportedNetworks/customContent/_subgraph-community.mdx index 29974c84ec03..2bb04ba78272 100644 --- a/website/src/supportedNetworks/customContent/_subgraph-community.mdx +++ b/website/src/supportedNetworks/customContent/_subgraph-community.mdx @@ -140,13 +140,22 @@ Deploying pushes your Subgraph to Subgraph Studio, where you can test it in the #### Optional: Verify locally with Graph Node Dev Mode -Run your Subgraph on your machine with [Graph Node Dev Mode (GND)](/subgraphs/developing/creating/graph-node-dev/), pointed at an RPC endpoint for {props.chainName}. From your Subgraph project directory: +Run your Subgraph on your machine with [Graph Node Dev Mode (GND)](/subgraphs/developing/creating/graph-node-dev/), pointed at an RPC endpoint for {props.chainName}. Hot reloading takes two processes: `gnd --watch` redeploys your Subgraph whenever the build output changes, but it does not compile your code. A build watcher recompiles your mappings, schema, and manifest when you edit them. + +In one terminal, from your Subgraph project directory, start the build watcher: + + + {`# Build once, then rebuild whenever Subgraph files change (or: gnd build --watch) +graph build --watch`} + + +In a second terminal, from the same directory, start GND: {`# Install gnd with the Graph CLI graph node install -# Build, deploy, and index your Subgraph locally, redeploying on changes +# Deploy and index the build output locally, redeploying on each new build gnd --ethereum-rpc ${props.networkId}:${props.rpcUrl} --watch`}