-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Indexers v2 #132
Merged
Merged
Indexers v2 #132
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
c56719d
Create indexers-v2 docker baseline
marcus-snx 5efd639
Fix package.json name
marcus-snx 5ef68ea
Create separate services for indexers
marcus-snx 1103017
Remove service
marcus-snx 4811eed
Add cannon config for base
marcus-snx 8676c18
Fix docker exiting when no migrations
marcus-snx 7907660
Make services explicit for indexers-v2
marcus-snx 412b0d5
Refactor logic
marcus-snx d9078aa
Handle Exceptions
marcus-snx ca9d08d
Fix arbitrum mainnet network config
marcus-snx 7275dc9
Consolidate contract data
marcus-snx cde7a83
Fix sdk version
marcus-snx 3623f3c
Create entrypoint bash script
marcus-snx 3136887
Add block range
marcus-snx a786170
Fix entrypoint script
marcus-snx 226f8a7
Update network config files
marcus-snx 992ea2b
Use sdk logger
marcus-snx 91ab6cf
Improve arg and config parsing
marcus-snx b7bb3ca
Add rate limit config
marcus-snx ea8f205
Fix base mainnet config file
marcus-snx 7889047
Add config for base mainnet parquet
marcus-snx 2976d5a
Set default config
marcus-snx 69d2ec6
Fix packages
marcus-snx 978b140
Add env var to entrypoint
marcus-snx 981affb
Add custom config to indexer service
marcus-snx ef9fb76
Remove migration deps in squid commands
marcus-snx b18a47e
Add parquet export support
marcus-snx d3eea6f
Fix patches paths in Dockerfile
marcus-snx fe8a67c
Remove required for config_name arg
marcus-snx 2eeaa5d
Patch gen tools for no archive url
marcus-snx 1bd0517
Fix blockrange
marcus-snx 5f1190a
Update indexers-v2/requirements.txt
Tburm 3646184
Remove contract renaming
marcus-snx fbe4130
Remove postgres target
marcus-snx b0dd1d3
Merge branch 'feat/indexers-v2' of github.com:Synthetixio/data into f…
marcus-snx 2f4b52d
Remove deprecated syncIntervalBlocks
marcus-snx bbeef8a
Merge branch 'main' into feat/indexers-v2
marcus-snx File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,18 +1,20 @@ | ||
import json | ||
import sys | ||
import os | ||
import argparse | ||
import yaml | ||
import logging | ||
from synthetix import Synthetix | ||
|
||
logging.basicConfig(level=logging.INFO) | ||
|
||
|
||
def save_abi(abi, contract_name): | ||
os.makedirs("abi", exist_ok=True) | ||
with open(f"abi/{contract_name}.json", "w") as file: | ||
json.dump(abi, file, indent=2) | ||
|
||
|
||
def create_squidgen_config(rpc_url, archive_url, contracts_info): | ||
def create_squidgen_config(rpc_url, archive_url, contracts_info, block_range): | ||
config = { | ||
"archive": archive_url, | ||
"finalityConfirmation": 1, | ||
|
@@ -27,6 +29,7 @@ def create_squidgen_config(rpc_url, archive_url, contracts_info): | |
contract_config = { | ||
"name": name, | ||
"address": address, | ||
"range": block_range, | ||
"abi": f"./abi/{name}.json", | ||
"events": True, | ||
"functions": False, | ||
|
@@ -99,21 +102,29 @@ def load_network_config(path): | |
|
||
contracts = [] | ||
|
||
if "cannon_config" in network_config: | ||
snx = Synthetix( | ||
provider_rpc=rpc_endpoint, | ||
network_id=network_config["network_id"], | ||
cannon_config=network_config["cannon_config"], | ||
) | ||
else: | ||
snx = Synthetix( | ||
provider_rpc=rpc_endpoint, | ||
network_id=network_config["network_id"], | ||
) | ||
|
||
block_range = {} | ||
block_range["from"] = network_config["range"].get("from", 0) | ||
if "to" in network_config["range"]: | ||
if network_config["range"]["to"] == "latest": | ||
block_range["to"] = snx.web3.eth.block_number | ||
else: | ||
block_range["to"] = network_config["range"]["to"] | ||
|
||
if "contracts_from_sdk" in network_config: | ||
contracts_from_sdk = network_config["contracts_from_sdk"] | ||
|
||
if "cannon_config" in network_config: | ||
snx = Synthetix( | ||
provider_rpc=rpc_endpoint, | ||
network_id=network_config["network_id"], | ||
cannon_config=network_config["cannon_config"], | ||
) | ||
else: | ||
snx = Synthetix( | ||
provider_rpc=rpc_endpoint, | ||
network_id=network_config["network_id"], | ||
) | ||
|
||
for contract in contracts_from_sdk: | ||
name = contract["name"] | ||
package = contract["package"] | ||
|
@@ -133,12 +144,14 @@ def load_network_config(path): | |
save_abi(contract_data["abi"], name) | ||
contracts.append({"name": name, "address": contract_data["address"]}) | ||
|
||
squidgen_config = create_squidgen_config(rpc_endpoint, archive_url, contracts) | ||
squidgen_config = create_squidgen_config( | ||
rpc_endpoint, archive_url, contracts, block_range | ||
) | ||
write_yaml(squidgen_config, "squidgen.yaml") | ||
|
||
squid_config = create_squid_config(args.network_name) | ||
write_yaml(squid_config, "squid.yaml") | ||
|
||
print( | ||
logging.info( | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. use |
||
f"squidgen.yaml, squid.yaml, and ABI files have been generated for {args.network_name}" | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can simplify a bit by leaving this out, and using
snx.logger.info