[wip] symbol lookups

[amitu]

So we have considered import desugaring, where each import can de-sugar into one or more module/symbol alias.

We have a definitions: std::collections::HashMap<fastn_unresolved::Symbol, fastn_unresolved::URD> and builtins: indexmap::IndexMap<String, fastn_resolved::Definition>.

When we are resolving a symbol, we can get the Definition:

pub struct Definition {
    pub symbol: Option<fastn_unresolved::Symbol>, // <package-name>/<module-name>#<definition-name>
    pub doc: Option<fastn_section::Span>,
    pub name: UR<fastn_section::Identifier, fastn_section::Identifier>,
    pub visibility: fastn_section::Visibility,
    pub inner: InnerDefinition,
}

From where we can find the symbol.

How To Do Relative Symbol Resolution

If a Definition internally refers to another relative symbol, foo, or foo.bar, how do we get the full symbol name of foo or foo.bar? foo can be either a module or symbol, how do we learn this?

Definition gives us symbol, from where we can obtain the current module[1].

In fastn_unresolved::ComponentInvocation::resolve(), we are passing the self.module as well. The self.module is the module name of the main module being rendered. A component invocation there can be for a component that is defined in another module, and when we are resolving that module, we need to resolve it with respect to that module. The Definition.symbol.module way will give every definition access to the module they are defined in.

[1]: if we are always using symbol to get the module, why not we store the module directly on the Definition?

Local Symbols

We can resolve global symbols using this method, but what about locals? A component has component arguments, and function can have function arguments, but inside function we can have a "stack of blocks", where each block may define some variables local to that block, so when we are resolving a symbol inside a component or function, we have to do lookups.

The component arguments are always prefixed by component name, so we do not have any ambiguity, if foo.bar is being resolved inside a component, and foo is the name of that component, then bar is the component argument. Components cannot internally define locally scoped variables, and all component variables are component arguments.

Should We Have Scope Stack?

What if each module had a "scope" object, which is a stack of scopes, we already have three scopes, the scope of symbols defined in a module, the scope of auto imports, this scope can be shared by all modules in current package, and the "global scope", this is the things that fastn provides globally, which currently only includes ftd.*, and maybe one can argue all the "basic types" eg integer belong to that scope.

For components we will have another scope which contains locals there, and for functions, we can have even more scopes (one scope per block with local definitions).

Upcoming Modules Feature

In parametric modules I discussed a module feature, that allows you to create local scope blocks within a single fastn module, and they can be arbitrarily nested, so a form of scope stack is relevant there as well.

Scope Shadowing And Named Scope

Imagine if all scopes can have names. The global scope, the outermost scope can be called kernel (or we could have called it ftd, but we have ftd.integer and integer with different meanings). Say the auto imports are auto-imports. And the current module scope is current-module. Each component defines a new scope named after the component. Same with functions arguments. Say inside a function, when we are creating a new block, there is some syntax to give the blocks a name:

-- integer x: 10

-- void foo():
integer x: 20

block-name#{ 
    integer x: 30;
    log(x + foo.x + block-name.x) ;; x is same as `current-module.x` since we are doing bottom to up search.
}

If each scope has a name, then we do not have to "shadow" a variable, and we can an non explicitly scoped variable will be looked up top of the scope stack to bottom, and if you want to explicitly reach out to a scope lowed down, you have to use the scope name. Since scope names themselves can be valid identifiers, maybe we will have to do bottom to top search for un-scoped variable.

Are Scopes Statically Associated with Each Definition / Component Invocation?

Parametric modules seems to require us have scope passed to definition, as the same definition can be present in different modules, and at that time we will have to decide if we want to clone the entire Definition, one per "new-module", or keep track of (Definition, Scope) pair everywhere.

For simplicity cloning seems superior, also: the UnResolved version of each Definition has to be cloned, as inside each "new-module", the definition has to be resolved again. So it's not just simplicity but correctness also requires us to clone.

How To Store Scope Stacks?

We can use id-arean::Arena<ScopeStack>, where type ScopeStack = Vec<(ScopeName, std::collections::HashMap<fastn_unresolved::Symbol, fastn_unresolved::URD>)>, where struct ScopeName(String). The id of the id-arena is small. If we have the scope stack maybe we do not need to store the module on Definition at all.