Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: automatically detect if genesis CAR is compressed #12885

Merged
merged 1 commit into from
Feb 11, 2025

Conversation

masih
Copy link
Member

@masih masih commented Feb 10, 2025

Related Issues

Proposed Changes

When --genesis path is set, automatically detect if the genesis file is ZSTD compressed and decompress it.

Additional Info

Checklist

Before you mark the PR ready for review, please make sure that:

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@masih masih force-pushed the masih/gen-auto-zstd branch from 45e8853 to d1d301d Compare February 10, 2025 11:33
@masih masih changed the title Automatically detect if genesis CAR is compressed feat: automatically detect if genesis CAR is compressed Feb 10, 2025
@github-actions github-actions bot dismissed their stale review February 10, 2025 11:33

PR title now matches the required format.

@masih masih force-pushed the masih/gen-auto-zstd branch 3 times, most recently from b9e5130 to 46b1c8e Compare February 10, 2025 11:40
@masih masih requested a review from rvagg February 10, 2025 11:41
@rjan90
Copy link
Contributor

rjan90 commented Feb 10, 2025

Did a test with this branch, and can confirm it works:

Before:

lotus daemon --genesis=/home/boost/lotus/build/genesis/mainnet.car.zst
2025-02-10T12:43:16.762+0100	INFO	main	lotus/daemon.go:230	lotus repo: /mnt/lotuschain/.lotus
2025-02-10T12:43:16.763+0100	INFO	paramfetch	[email protected]/paramfetch.go:209	Parameter file /var/tmp/filecoin-proof-parameters/v28-empty-sector-update-merkletree-poseidon_hasher-8-0-0-92180959e1918d26350b8e6cfe217bbdd0a2d8de51ebec269078b364b715ad63.vk is ok
2025-02-10T12:43:16.763+0100	INFO	paramfetch	[email protected]/paramfetch.go:209	Parameter file /var/tmp/filecoin-proof-parameters/v28-empty-sector-update-merkletree-poseidon_hasher-8-8-0-3b7f44a9362e3985369454947bc94022e118211e49fd672d52bec1cbfd599d18.vk is ok
--------
ERROR: initializing node: starting node: could not build arguments for function "github.com/filecoin-project/lotus/node".ConfigFullNode.StartListening.func10 (/home/boost/lotus/node/modules/lp2p/addrs.go:85): failed to build host.Host: could not build arguments for function "reflect".makeFuncStub (/usr/local/go/src/reflect/asm_amd64.s:28): failed to build lp2p.BaseIpfsRouting: could not build arguments for function "reflect".makeFuncStub (/usr/local/go/src/reflect/asm_amd64.s:28): failed to build dtypes.NetworkName: could not build arguments for function "reflect".makeFuncStub (/usr/local/go/src/reflect/asm_amd64.s:28): failed to build dtypes.AfterGenesisSet: received non-nil error from function "reflect".makeFuncStub (/usr/local/go/src/reflect/asm_amd64.s:28): getting expected genesis failed: loading genesis car file failed: invalid header: malformed stream: invalid appearance of int token; expected map key

After:

lotus daemon --genesis=/home/boost/lotus/build/genesis/mainnet.car.zst
2025-02-10T12:40:04.580+0100	INFO	lotus	lotus/daemon.go:230	lotus repo: /mnt/lotuschain/.lotus
2025-02-10T12:40:04.588+0100	INFO	paramfetch	[email protected]/paramfetch.go:209	Parameter file /var/tmp/filecoin-proof-parameters/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-0-0-0170db1f394b35d995252228ee359194b13199d259380541dc529fb0099096b0.vk is ok
2025-02-10T12:40:04.588+0100	INFO	paramfetch	[email protected]/paramfetch.go:209	Parameter file /var/tmp/filecoin-proof-parameters/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-0-0-0cfb4f178bbb71cf2ecfcd42accce558b27199ab4fb59cb78f2483fe21ef36d9.vk is ok
--------
25-02-10T12:40:18.013+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.622022872, "height": "4696040", "age": 18.013856694}
2025-02-10T12:40:18.016+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.623518005, "height": "4696040", "age": 18.016124076}
2025-02-10T12:40:18.016+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.624685525, "height": "4696040", "age": 18.01613177}
2025-02-10T12:40:18.016+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.624374797, "height": "4696040", "age": 18.01615072}
2025-02-10T12:40:18.018+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.626626394, "height": "4696040", "age": 18.018491273}
2025-02-10T12:40:18.022+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.631333797, "height": "4696040", "age": 18.022833674}

@masih masih force-pushed the masih/gen-auto-zstd branch 2 times, most recently from f126b2c to c77bc27 Compare February 10, 2025 11:59
build/genesis.go Outdated Show resolved Hide resolved
cli/lotus/daemon.go Outdated Show resolved Hide resolved
Copy link
Member

@rvagg rvagg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌 love it, we need more of this simple QoL sanity checking

When `--genesis` path is set, automatically detect if the genesis file
is ZSTD compressed and decompress it.
@masih masih force-pushed the masih/gen-auto-zstd branch from c77bc27 to ccc9751 Compare February 11, 2025 10:07
@masih masih merged commit 48d0ab2 into master Feb 11, 2025
83 checks passed
@masih masih deleted the masih/gen-auto-zstd branch February 11, 2025 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🎉 Done
Development

Successfully merging this pull request may close these issues.

3 participants