[Feature] Support importing local and remote programs.

[evan-schott]

tldr

• Now super easy to develop leo projects that make external calls to on-chain and local programs!
  • Battleship v2 - using local dependencies
  • Nested network dependencies
  • Mix of nested local and nested network dependencies
• Dependencies must be written in program.json as well as main.leo
• We used to write import credits.leo, but now we have changed it to import credits.aleo for consistency.
• In order for external structs to work, they must be defined exactly the same in both the external file and the local file.

Motivation

• We want a richer dependency resolution system in Leo. Leo users should be able to import programs from chain/local file system as opposed to requiring .leo source files.

Components:

1. Specify import configurations in program.json
2. leo build leverages retriever module to pull programs from network/FS
3. Generate leo.lock file to house the dependency configuration
4. disassembler.rs to convert Aleo instructions into valid stub
5. Type checking modification to maintain soundness
6. Codegen modification to print nested imports properly

Retriever Design

Modifications to ~/.aleo file structure

• .aleo/registry:

  • .aleo/registry/git : Where .aleo files from github.com are stored
  • .aleo/registry/testnet3: Where .aleo files from the network are stored
  • Note that new folders can be added for different networks like ./mainnet

• Suppose building leo project dex and call leo build with dex/program.json set to:

{
  "program": "dex.aleo",
  "version": "0.0.0",
  "description": "",
  "license": "MIT",
  "dependencies": [
    {
	  "name": "credits.aleo",
	  "location": "network",
		"network": "testnet3"
    },
	{
		"name": "foo.aleo",
		"location": "git",
		"url": "https://github.com/alice/foo.git"
		"branch": "bar"
		"rev": "3ab873bd89de"
		},
    {
	  "name": "board.aleo",
	  "location": "local",
	  "path": "../board", 
    },
  ]
}

Specification and example flow

1. Retrieve imports based on configurations specified in dex/program.json:
   • Must keep track of nested programs trying to build, and throw error for circular dependencies
   • Location variations:
   • "location": "network":
     • Check if credits.aleo cached already in ~/.aleo/registry/testnet3/credits.aleo
     • Retrieve from network:
       • Store copy in `~/.aleo/registry/testnet3/credits.aleo``
       • Update leo.lock
   • "location": "local":
     • Recursively build the directory (must read its program.json to see its dependencies)
     • Cache in .aleo and update leo.lock
   • "location": "git": TODO
     • Example: "url": "<https://github.com/alice/foo.git>"
     • Check if board.aleo cached already as ~/.aleo/registry/git/pranav/foo/board.aleo
     • Otherwise retrieve from network and cache
     • Allow them to specify based on commit + branch
2. Move a copy of the retrieved import(s) into dex/build/imports/
3. Convert all imported programs to AST before start parsing main.leo

[evan-schott]

I resolved the 2 file.rs changes (and comment.out as that error is a consequence of how I changed the parsing of program_ids in file.rs) as I will be reverting all of the file.rs` changes related to stubs (since we are not going to allow explicit specification of stubs) and keeping just those parser changes related to importing.

[d0cd]

LGTM! Amazing work @evan-schott!