feat(store-sync): add util to fetch snapshot from indexer with SQL API (

#2996) Co-authored-by: Andy Cernera <[email protected]> Co-authored-by: Ori Pomerantz <[email protected]>
latticexyz · Sep 18, 2024 · 8b4110e · 8b4110e
1 parent 83fc93c
commit 8b4110e
Show file tree

Hide file tree

Showing 29 changed files with 1,662 additions and 5 deletions.
diff --git a/.changeset/soft-boats-protect.md b/.changeset/soft-boats-protect.md
@@ -0,0 +1,6 @@
+---
+"@latticexyz/protocol-parser": patch
+"@latticexyz/store-sync": patch
+---
+
+Added `store-sync` helper libraries to interact with the indexer's experimental SQL API endpoint. Documentation is available at [https://mud.dev/indexer/sql](https://mud.dev/indexer/sql).
diff --git a/docs/components/common-text/FilterTypes.mdx b/docs/components/common-text/FilterTypes.mdx
@@ -0,0 +1,13 @@
+import { Callout } from "nextra/components";
+
+<Callout type="info">
+MUD initial data hydration, and therefore filtering, comes in two flavors: [SQL](/indexer/sql) and [generic](/guides/hello-world/filter-sync).
+Note that this is for the initial hydration, currently limits on on-going synchronization are limited to [the generic method](/guides/hello-world/filter-sync).
+
+|                      | SQL                                                                                                                                                | Generic                                                                                                                                                   |
+| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Filtering            | Can filter on most SQL functions                                                                                                                   | Can only filter on tables and the first two key fields (limited by [`eth_getLogs`](https://ethereum.github.io/execution-apis/api-documentation/) filters) |
+| Availability         | [Redstone](https://redstone.xyz/docs/network-info), [Garnet](https://garnetchain.com/docs/network-info), or elsewhere if you run your own instance | Any EVM chain                                                                                                                                             |
+| Security assumptions | The indexer instance returns accurate information                                                                                                  | The endpoint returns accurate information (same assumption as any other blockchain app)                                                                   |
+
+</Callout>
diff --git a/docs/pages/guides/hello-world/filter-sync.mdx b/docs/pages/guides/hello-world/filter-sync.mdx
@@ -1,10 +1,25 @@
 import { CollapseCode } from "../../../components/CollapseCode";
+import FilterTypes from "../../../components/common-text/FilterTypes.mdx";
 
 # Filter data synchronization
 
 In this tutorial you modify `networkSetup.ts` to filter the information you synchronize.
 Filtering information this way allows you to reduce the use of network resources and makes loading times faster.
 
+<FilterTypes />
+
+<details>
+
+<summary>Why are only the first two key fields available for filtering?</summary>
+
+Ethereum log entries can have [up to four indexed fields](https://www.evm.codes/?fork=cancun#a4).
+However, Solidity only supports [three indexed fields](https://www.alchemy.com/overviews/solidity-events) because the first indexed field is used for the event name and type.
+In MUD, [this field](https://github.com/latticexyz/mud/blob/main/packages/store/src/IStoreEvents.sol) specifies whether [a new record is created](https://github.com/latticexyz/mud/blob/main/packages/store/src/IStoreEvents.sol#L26-L32), a record is changed (either [static fields](https://github.com/latticexyz/mud/blob/main/packages/store/src/IStoreEvents.sol#L43) or [dynamic fields](https://github.com/latticexyz/mud/blob/main/packages/store/src/IStoreEvents.sol#L56-L64)), or [a record is deleted](https://github.com/latticexyz/mud/blob/main/packages/store/src/IStoreEvents.sol#L71).
+The second indexed fields is always the table's [resource ID](/world/resource-ids).
+This leaves two fields for key fields.
+
+</details>
+
 ## Setup
 
 To see the effects of filtering we need a table with entries to filter. To get such a table:

diff --git a/docs/pages/indexer/_meta.js b/docs/pages/indexer/_meta.js
@@ -2,5 +2,6 @@ export default {
   "using": "Using the Indexer",
   "sqlite": "SQLite Indexer",
   "postgres-event-only": "PostgreSQL for events",
-  "postgres-decoded": "PostgreSQL for data (and events)",  
+  "postgres-decoded": "PostgreSQL for data (and events)",
+  sql: "SQL API (Experimental)",
 };