Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement "stateless" modules #3663

Merged
merged 140 commits into from
Dec 16, 2023
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
140 commits
Select commit Hold shift + click to select a range
8a56425
feat: implement pure libraries
charles-cooper Nov 7, 2023
0625bbb
fix topsort
charles-cooper Nov 7, 2023
684a83d
fix a typo
charles-cooper Nov 7, 2023
3ef38c4
introduce ModuleInfo
charles-cooper Nov 8, 2023
e3edb7e
fix a circular import
charles-cooper Nov 8, 2023
eca6379
change a docstring
charles-cooper Nov 8, 2023
bb37ee2
`at()` to convert ModuleT to InterfaceT
charles-cooper Nov 8, 2023
2038549
fix a bug
charles-cooper Nov 8, 2023
3f38b43
feat: module.at(<address>)
charles-cooper Nov 8, 2023
2caaf9d
Merge branch 'master' into feat/libraries
charles-cooper Nov 9, 2023
aba6558
fix some bugs
charles-cooper Nov 9, 2023
febaa96
fix some tests
charles-cooper Nov 9, 2023
19404e9
detect import cycles
charles-cooper Nov 10, 2023
21a04c3
remove `.name` from module ast
charles-cooper Nov 11, 2023
52943e1
abstract out a common exception rewriting pattern
charles-cooper Nov 11, 2023
4a8cc12
remove a note
charles-cooper Nov 17, 2023
0a786f7
rollback `module.at()` method
charles-cooper Nov 20, 2023
b61176e
fix some tests
charles-cooper Nov 21, 2023
6c7f795
refactor builtin interfaces to `.vyi` files
charles-cooper Nov 23, 2023
c923714
wip: .vyi files
charles-cooper Nov 23, 2023
924980a
implement ellipsis in interfaces
charles-cooper Nov 28, 2023
e9da1f2
whitespace
charles-cooper Dec 6, 2023
4a3cfd2
fix getter generation in ABI
charles-cooper Dec 6, 2023
6a7e603
rename "type" metadata to "func_type" on function declaration objects
charles-cooper Dec 7, 2023
9171934
Merge branch 'master' into feat/libraries
charles-cooper Dec 7, 2023
6c81142
fix some interface tests
charles-cooper Dec 7, 2023
e7e37b8
fix some more tests
charles-cooper Dec 7, 2023
1b7162b
revert change enabling `Bytes[...]`
charles-cooper Dec 10, 2023
26fb9e0
fix old usage of reachable_internal_functions
charles-cooper Dec 10, 2023
1e2a235
fix another type error
charles-cooper Dec 10, 2023
c58bfa0
fix sqrt
charles-cooper Dec 10, 2023
467720b
ensure _ir_info is populated whether or not functions are reachable
charles-cooper Dec 10, 2023
c6eba5e
fix a bad test
charles-cooper Dec 10, 2023
9c537cc
fix some interfaces in tests
charles-cooper Dec 10, 2023
25563f6
fix a bad type dispatch
charles-cooper Dec 10, 2023
9838ad6
fix call to validate_semantics
charles-cooper Dec 10, 2023
90731a2
fix InterfaceT.from_Module
charles-cooper Dec 10, 2023
a86cc02
set search path correctly in nested imports
charles-cooper Dec 10, 2023
82cb5bd
fix some tests
charles-cooper Dec 10, 2023
0a3d618
fix another small test
charles-cooper Dec 10, 2023
c6d8892
fix lint
charles-cooper Dec 10, 2023
88843e9
fix some calls to validate_semantics
charles-cooper Dec 10, 2023
2f739d0
fix mypy
charles-cooper Dec 10, 2023
0ce2a58
add a todo
charles-cooper Dec 10, 2023
ba50a83
fix an old key
charles-cooper Dec 10, 2023
dc506a8
move test_interfaces.py to codegen/ tests
charles-cooper Dec 10, 2023
ecb3841
remove dead code
charles-cooper Dec 10, 2023
5bb66b5
remove dead package pytest-rerunfailures
charles-cooper Dec 10, 2023
3d5e3b1
add a note
charles-cooper Dec 10, 2023
ad78f07
remove dead function
charles-cooper Dec 10, 2023
7ebd09c
refactor: move `parse_*` functions, remove vyper.ast.annotation
charles-cooper Dec 10, 2023
a0ff2b8
refactor: CompilerInput.from_string
charles-cooper Dec 10, 2023
84a33db
implement eq/hash on ModuleT
charles-cooper Dec 10, 2023
84f2298
remove dead comment
charles-cooper Dec 10, 2023
21abfb6
fix some bad imports
charles-cooper Dec 10, 2023
2a9db07
fix lint
charles-cooper Dec 10, 2023
0b78923
remove `remove_unused_statements()`
charles-cooper Dec 10, 2023
269a9cf
refactor cycle detection
charles-cooper Dec 10, 2023
b0ac484
remove expand_annotated_ast
charles-cooper Dec 10, 2023
a95000d
fix analysis on imported modules
charles-cooper Dec 10, 2023
50f388e
small fixes
charles-cooper Dec 10, 2023
9b5bff2
factor out parse_and_fold_ast
charles-cooper Dec 10, 2023
a43beff
fix mypy
charles-cooper Dec 10, 2023
487e29b
fix global namespace
charles-cooper Dec 10, 2023
abbe697
fix generate_inline_function
charles-cooper Dec 10, 2023
9509dc9
Merge branch 'master' into feat/libraries
charles-cooper Dec 10, 2023
fa4a30f
fix tokenization of comments in lark grammar
charles-cooper Dec 10, 2023
292d085
rename storage layout test directory
charles-cooper Dec 10, 2023
5b8847e
relax import restrictions
charles-cooper Dec 10, 2023
b26c00a
improve ModuleNotFound error
charles-cooper Dec 10, 2023
e4693a0
fix name/alias resolution
charles-cooper Dec 10, 2023
93401f3
add library test to test_compile_files
charles-cooper Dec 10, 2023
4295dcc
missing import
charles-cooper Dec 10, 2023
373abd6
update interfaces output
charles-cooper Dec 10, 2023
663dc1e
change function labels in IR
charles-cooper Dec 10, 2023
7ac2f62
add a note
charles-cooper Dec 10, 2023
d297418
fix: use codegen.core._freshname to generate self call labels
charles-cooper Dec 10, 2023
34b45c8
make compilation targets adhere to outputSelection
charles-cooper Dec 10, 2023
5c0d3a7
use orderedset for json inputs
charles-cooper Dec 10, 2023
9902baf
add library to json tests
charles-cooper Dec 10, 2023
9bdf12c
remove dead code
charles-cooper Dec 10, 2023
64f6840
fix lint
charles-cooper Dec 10, 2023
42dfd2b
fix lint
charles-cooper Dec 10, 2023
ba11cff
fix a signature
charles-cooper Dec 11, 2023
c7ea5c9
disambiguate types with same name across different modules
charles-cooper Dec 11, 2023
418eb97
fix some small bugs
charles-cooper Dec 11, 2023
e64b52c
update some hardcoded asm
charles-cooper Dec 11, 2023
f5ff470
fix unassigned variable
charles-cooper Dec 11, 2023
c86d267
fix lint
charles-cooper Dec 11, 2023
0ac1043
make module argument optional for structs
charles-cooper Dec 11, 2023
5ebb252
fix some updates to json cli api
charles-cooper Dec 11, 2023
d880e58
add libraries test file
charles-cooper Dec 11, 2023
34052b0
roll back changes require threading around the module ast
charles-cooper Dec 11, 2023
15ad3cc
fix lint
charles-cooper Dec 11, 2023
91642f0
add InputBundle._normalize_path
charles-cooper Dec 11, 2023
4db4928
rename ModuleInfo.module to ModuleInfo.module_t
charles-cooper Dec 11, 2023
028d96c
fix source_id threading
charles-cooper Dec 11, 2023
8d7a5dd
enrich CompilerInput
charles-cooper Dec 11, 2023
0a37e1c
change compile_code api from contract_name= to contract_path=
charles-cooper Dec 11, 2023
8b477eb
introduce source_id= back into compile_code
charles-cooper Dec 11, 2023
64a5379
add compile_from_file_input
charles-cooper Dec 11, 2023
39575e8
fix bunch of bugs in path resolution of imports
charles-cooper Dec 11, 2023
9ef6ea7
fix compile for regular files
charles-cooper Dec 11, 2023
b64035d
add support for attribute access on modules/interfaces: structs and e…
charles-cooper Dec 11, 2023
58940a0
allow string input to CompilerData
charles-cooper Dec 11, 2023
f34d7d1
remove ability to `implements: ModuleT`
charles-cooper Dec 11, 2023
be07dfd
fix small type bug
charles-cooper Dec 11, 2023
88250cc
fix a bad function call
charles-cooper Dec 11, 2023
183aada
fix a bug in _normalize_path
charles-cooper Dec 11, 2023
0a84ef2
fix another bad function call
charles-cooper Dec 11, 2023
3d2c95f
handle self calls in statements
charles-cooper Dec 12, 2023
2e32ba8
fix transitive imports
charles-cooper Dec 12, 2023
5476076
trap calls to external functions in libraries
charles-cooper Dec 12, 2023
bfde0cb
add more tests for libraries
charles-cooper Dec 12, 2023
7aa80e3
new scheme for IR identifiers -- use function ID
charles-cooper Dec 12, 2023
91c16e5
update selector table stability tests
charles-cooper Dec 12, 2023
5a9eea7
fix asm tests
charles-cooper Dec 12, 2023
59922f3
fix lint, mypy
charles-cooper Dec 12, 2023
a734b09
hygiene: raise proper exception
charles-cooper Dec 12, 2023
dd1ea14
test imported structs
charles-cooper Dec 12, 2023
c7584bd
add a couple more tests for libraries
charles-cooper Dec 12, 2023
76f3abb
fix a regression
charles-cooper Dec 12, 2023
48a76b4
handle some errors better
charles-cooper Dec 12, 2023
1fafca6
fix mypy
charles-cooper Dec 12, 2023
a441e2f
remove a dead variable
charles-cooper Dec 12, 2023
6e1a05e
update a test name
charles-cooper Dec 12, 2023
94ff82b
change an argument name
charles-cooper Dec 12, 2023
6edcbdd
reject duplicate imports
charles-cooper Dec 12, 2023
d9b9f3b
fix some tests
charles-cooper Dec 13, 2023
0b64abe
rename test file
charles-cooper Dec 13, 2023
255a93a
refactor: move InterfaceT into vyper/semantics/types/module.py
charles-cooper Dec 13, 2023
9ec83d4
address review comments
charles-cooper Dec 13, 2023
74c630e
remove unneeded F401
charles-cooper Dec 13, 2023
132fde0
add library code eliminator test
charles-cooper Dec 14, 2023
64bc3a5
make nested libraries work
charles-cooper Dec 15, 2023
dcb55b4
refactor a side effect
charles-cooper Dec 15, 2023
d26f401
fix missing import
charles-cooper Dec 15, 2023
bea05ca
add test for imported slice
charles-cooper Dec 15, 2023
7066688
remove stray print
charles-cooper Dec 16, 2023
b0c6684
add a note on Module.__eq__
charles-cooper Dec 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tests/unit/cli/vyper_compile/test_compile_files.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ def foo() -> FooStruct:

@external
def bar(a: address) -> FooStruct:
return {}(a).bar()
return {}.at(a).bar()
"""

BAR_CODE = """
Expand Down
13 changes: 8 additions & 5 deletions vyper/ast/annotation.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@ def __init__(
modification_offsets: Optional[ModificationOffsets],
tokens: asttokens.ASTTokens,
source_id: int,
contract_name: Optional[str],
module_path: Optional[str] = None,
):
self._tokens = tokens
self._source_id = source_id
self._contract_name = contract_name
self._module_path = module_path
self._source_code: str = source_code
self.counter: int = 0
self._modification_offsets = {}
Expand Down Expand Up @@ -83,7 +83,8 @@ def _visit_docstring(self, node):
return node

def visit_Module(self, node):
node.name = self._contract_name
node.path = self._module_path
node.source_id = self._source_id
return self._visit_docstring(node)

def visit_FunctionDef(self, node):
Expand Down Expand Up @@ -250,7 +251,7 @@ def annotate_python_ast(
source_code: str,
modification_offsets: Optional[ModificationOffsets] = None,
source_id: int = 0,
contract_name: Optional[str] = None,
module_path: Optional[str] = None,
) -> python_ast.AST:
"""
Annotate and optimize a Python AST in preparation conversion to a Vyper AST.
Expand All @@ -270,7 +271,9 @@ def annotate_python_ast(
"""

tokens = asttokens.ASTTokens(source_code, tree=cast(Optional[python_ast.Module], parsed_ast))
visitor = AnnotatingVisitor(source_code, modification_offsets, tokens, source_id, contract_name)
visitor = AnnotatingVisitor(
source_code, modification_offsets, tokens, source_id, module_path=module_path
)
visitor.visit(parsed_ast)

return parsed_ast
3 changes: 2 additions & 1 deletion vyper/ast/nodes.py
Original file line number Diff line number Diff line change
Expand Up @@ -589,7 +589,8 @@ def __contains__(self, obj):


class Module(TopLevel):
__slots__ = ()
# metadata
__slots__ = ("path", "source_id")

def replace_in_tree(self, old_node: VyperNode, new_node: VyperNode) -> None:
"""
Expand Down
5 changes: 3 additions & 2 deletions vyper/ast/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ def parse_to_ast(*args: Any, **kwargs: Any) -> vy_ast.Module:
def parse_to_ast_with_settings(
source_code: str,
source_id: int = 0,
contract_name: Optional[str] = None,
module_path: Optional[str] = None,
add_fn_node: Optional[str] = None,
) -> tuple[Settings, vy_ast.Module]:
"""
Expand Down Expand Up @@ -53,7 +53,8 @@ def parse_to_ast_with_settings(
fn_node.body = py_ast.body
fn_node.args = python_ast.arguments(defaults=[])
py_ast.body = [fn_node]
annotate_python_ast(py_ast, source_code, class_types, source_id, contract_name)

annotate_python_ast(py_ast, source_code, class_types, source_id, module_path=module_path)

# Convert to Vyper AST.
module = vy_ast.get_node(py_ast)
Expand Down
6 changes: 2 additions & 4 deletions vyper/builtins/_utils.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
from vyper.ast import parse_to_ast
from vyper.codegen.context import Context
from vyper.codegen.global_context import GlobalContext
from vyper.codegen.stmt import parse_body
from vyper.semantics.analysis.local import FunctionNodeVisitor
from vyper.semantics.namespace import Namespace, override_global_namespace
from vyper.semantics.types.function import ContractFunctionT, FunctionVisibility, StateMutability
from vyper.semantics.types.module import ModuleT
Dismissed Show dismissed Hide dismissed


def _strip_source_pos(ir_node):
Expand All @@ -29,9 +29,7 @@
# annotate the AST as side effects.
FunctionNodeVisitor(ast_code, ast_code.body[0], namespace)

new_context = Context(
vars_=variables, global_ctx=GlobalContext(), memory_allocator=memory_allocator
)
new_context = Context(vars_=variables, module_ctx=ModuleT(), memory_allocator=memory_allocator)
Fixed Show fixed Hide fixed
generated_ir = parse_body(ast_code.body[0].body, new_context)
# strip source position info from the generated_ir since
# it doesn't make any sense (e.g. the line numbers will start from 0
Expand Down
8 changes: 4 additions & 4 deletions vyper/codegen/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ def __repr__(self):
class Context:
def __init__(
self,
global_ctx,
module_ctx,
memory_allocator,
vars_=None,
forvars=None,
Expand All @@ -60,7 +60,7 @@ def __init__(
self.vars = vars_ or {}

# Global variables, in the form (name, storage location, type)
self.globals = global_ctx.variables
self.globals = module_ctx.variables

# Variables defined in for loops, e.g. for i in range(6): ...
self.forvars = forvars or {}
Expand All @@ -75,8 +75,8 @@ def __init__(
# Whether we are currently parsing a range expression
self.in_range_expr = False

# store global context
self.global_ctx = global_ctx
# store module context
self.module_ctx = module_ctx

# full function type
self.func_t = func_t
Expand Down
47 changes: 24 additions & 23 deletions vyper/codegen/expr.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@
TupleT,
)
from vyper.semantics.types.bytestrings import _BytestringT
from vyper.semantics.types.function import ContractFunctionT, MemberFunctionT
Dismissed Show dismissed Hide dismissed
from vyper.semantics.types.shortcuts import BYTES32_T, UINT256_T

Check notice

Code scanning / CodeQL

Cyclic import Note

Import of module
vyper.semantics.types.shortcuts
begins an import cycle.
from vyper.utils import (
DECIMAL_DIVISOR,
bytes_to_int,
Expand Down Expand Up @@ -662,39 +663,39 @@
if function_name in DISPATCH_TABLE:
return DISPATCH_TABLE[function_name].build_IR(self.expr, self.context)

# Struct constructors do not need `self` prefix.
elif isinstance(self.expr._metadata["type"], StructT):
args = self.expr.args
if len(args) == 1 and isinstance(args[0], vy_ast.Dict):
return Expr.struct_literals(args[0], self.context, self.expr._metadata["type"])
func_type = self.expr.func._metadata["type"]
return_type = self.expr._metadata["type"]
Fixed Show fixed Hide fixed

# Interface assignment. Bar(<address>).
elif isinstance(self.expr._metadata["type"], InterfaceT):
(arg0,) = self.expr.args
arg_ir = Expr(arg0, self.context).ir_node
# Struct constructors do not need `self` prefix.
if isinstance(return_type, StructT):
args = self.expr.args
if len(args) == 1 and isinstance(args[0], vy_ast.Dict):
return Expr.struct_literals(args[0], self.context, self.expr._metadata["type"])

assert arg_ir.typ == AddressT()
arg_ir.typ = self.expr._metadata["type"]
# Interface assignment. Bar(<address>).
if isinstance(return_type, InterfaceT):
(arg0,) = self.expr.args
arg_ir = Expr(arg0, self.context).ir_node

return arg_ir
assert arg_ir.typ == AddressT()
arg_ir.typ = self.expr._metadata["type"]

elif isinstance(self.expr.func, vy_ast.Attribute) and self.expr.func.attr == "pop":
return arg_ir

if isinstance(func_type, MemberFunctionT) and self.expr.func.attr == "pop":
# TODO consider moving this to builtins
darray = Expr(self.expr.func.value, self.context).ir_node
assert len(self.expr.args) == 0
assert isinstance(darray.typ, DArrayT)
return pop_dyn_array(darray, return_popped_item=True)

elif (
# TODO use expr.func.type.is_internal once
# type annotations are consistently available
isinstance(self.expr.func, vy_ast.Attribute)
and isinstance(self.expr.func.value, vy_ast.Name)
and self.expr.func.value.id == "self"
):
return self_call.ir_for_self_call(self.expr, self.context)
else:
return external_call.ir_for_external_call(self.expr, self.context)
if isinstance(func_type, ContractFunctionT):
if self.expr.func._metadata["type"].is_internal:
return self_call.ir_for_self_call(self.expr, self.context)
else:
return external_call.ir_for_external_call(self.expr, self.context)

raise CompilerPanic("unreachable", self.expr)

def parse_List(self):
typ = self.expr._metadata["type"]
Expand Down
6 changes: 3 additions & 3 deletions vyper/codegen/function_definitions/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@
from vyper.codegen.core import check_single_exit
from vyper.codegen.function_definitions.external_function import generate_ir_for_external_function
from vyper.codegen.function_definitions.internal_function import generate_ir_for_internal_function
from vyper.codegen.global_context import GlobalContext
from vyper.codegen.ir_node import IRnode
from vyper.codegen.memory_allocator import MemoryAllocator
from vyper.exceptions import CompilerPanic
from vyper.semantics.types import VyperType
from vyper.semantics.types.function import ContractFunctionT
from vyper.semantics.types.module import ModuleT
from vyper.utils import MemoryPositions, calc_mem_gas, mkalphanum


Expand Down Expand Up @@ -94,7 +94,7 @@ class InternalFuncIR(FuncIR):

# TODO: should split this into external and internal ir generation?
def generate_ir_for_function(
code: vy_ast.FunctionDef, global_ctx: GlobalContext, is_ctor_context: bool = False
code: vy_ast.FunctionDef, module_ctx: ModuleT, is_ctor_context: bool = False
) -> FuncIR:
"""
Parse a function and produce IR code for the function, includes:
Expand Down Expand Up @@ -126,7 +126,7 @@ def generate_ir_for_function(

context = Context(
vars_=None,
global_ctx=global_ctx,
module_ctx=module_ctx,
memory_allocator=memory_allocator,
constancy=Constancy.Mutable if func_t.is_mutable else Constancy.Constant,
func_t=func_t,
Expand Down
32 changes: 0 additions & 32 deletions vyper/codegen/global_context.py

This file was deleted.

Loading
Loading