Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates explanation of modules #369

Merged
merged 3 commits into from
Dec 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions _books/ion-1-1/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,11 @@
- [System macros](macros/system_macros.md)
- [Modules](modules.md)
- [Defining modules](modules/defining_modules.md)
- [The encoding module](modules/encoding_module.md)
- [Directives](modules/directives.md)
- [Local modules](modules/local_modules.md)
- [Encoding modules](modules/encoding_modules.md)
- [System module](modules/system_module.md)
- [Shared modules](modules/shared_modules.md)
- [Inner modules](modules/inner_modules.md)
- [The system module](modules/system_module.md)
- [Binary encoding](binary/encoding.md)
- [Encoding primitives](binary/primitives.md)
- [`FlexUInt`](binary/primitives/flex_uint.md)
Expand Down
4 changes: 2 additions & 2 deletions _books/ion-1-1/src/glossary.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Glossary

**active encoding module**<br/>
The _encoding module_ whose symbol table and macro table are available in the current _segment_ of an Ion _document_.
The active encoding module is set by a _directive_.
An _encoding module_ whose symbol table and macro table are available in the current _segment_ of an Ion _document_.
The sequence of active encoding modules is set by an _encoding directive_.

**argument**<br/>
The sub-expression(s) within a macro invocation, corresponding to exactly one of the macro's parameters.
Expand Down
45 changes: 26 additions & 19 deletions _books/ion-1-1/src/modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,16 @@ Ion 1.1 also introduces the concept of a _module_, an organizational unit that h
> [!TIP]
> You can think of an Ion 1.0 symbol table as a module with an empty macro table.

In Ion 1.1, each stream has an [encoding module](modules/encoding_module.md)—the active `(symbol table, macro table)` pair that is being used to encode the stream.
In Ion 1.1, each stream has an [encoding module sequence](modules/encoding_modules.md)—
a collection of modules whose symbols and macros are being used to encode the current segment.

## Module interface

The interface to a module consists of:

* its _spec version_, denoting the Ion version used to define the module
* its _exported symbols_, an array of strings denoting symbol content
* its _exported macros_, an array of `<name, macro>` pairs, where all names are unique identifiers (or null).
* its _exported macros_, an array of `<name, macro>` pairs, where all names (where specified) are unique identifiers

The spec version is external to the module body and the precise way it is determined depends on the type of module being defined. This is explained in further detail in [Module Versioning](#module-versioning).

Expand All @@ -27,30 +28,30 @@ by the `symbols` field of a shared symbol table.
The exported macro array is denoted by the module’s `macro_table` clause, with addresses
allocated to macros or macro bindings in the order they are declared.

The exported symbols and exported macros are defined in the [module body](body.md).
The exported symbols and exported macros are defined in the [module body](modules/defining_modules.md).


## Types of modules

There are multiple types of modules.
All modules share the same interface, but vary in their implementation in order to support a variety of different use cases.

| Module Type | Purpose |
|:----------------------------------------------|:---------------------------------------------------------------|
| [Encoding Module](modules/encoding_module.md) | Defining the local encoding context |
| [System Module](modules/system_module.md) | Defining system symbols and macros |
| [Inner Module](modules/inner_modules.md) | Organizing symbols and macros and limiting the scope of macros |
| [Shared Module](modules/shared_modules.md) | Defining symbols and macros outside of the data stream |
| Module Type | Purpose |
|:------------------------------------------------|:----------------------------------------------------------------|
| [Local Modules](modules/local_modules.md) | Organizing symbols and macros within a scope |
| [Shared Modules](modules/shared_modules.md) | Defining reusable symbols and macros outside of the data stream |
| [System Modules](modules/system_module.md) | Defining system symbols and macros |
| [Encoding Modules](modules/encoding_modules.md) | Encoding the current stream segment |


## Module versioning

Every module definition has a _spec version_ that determines the syntax and semantics of the module body.
A module’s spec version is expressed in terms of a specific Ion version; the meaning of the module is as defined by that version of the Ion specification.
A module’s spec version is expressed in terms of a specific Ion version;
the meaning of the module is as defined by that version of the Ion specification.

The spec version for an encoding module is implicitly derived from the Ion version of its containing segment.
The spec version for a local module is inherited from its parent scope, which may be the stream itself.
The spec version for a shared module is denoted via a required annotation.
The spec version of an inner module is always the same as its containing module.
The spec version of a system module is the Ion version in which it was specified.

To ensure that all consumers of a module can properly understand it, a module can only import
Expand All @@ -62,29 +63,35 @@ This allows the module to be serialized using any version of Ion, and its meanin

```ion
$ion_shared_module::
$ion_1_1::("com.example.symtab" 3
(symbol_table ...)
(macro_table ...))
$ion_1_1::
("com.example.symtab" 3
(symbol_table ...)
(macro_table ...))
```

The spec version of an encoding module is always the same as the Ion version of its enclosing segment.
The spec version of a local module is always the same as the spec version of its enclosing scope.
If the local module is defined at the top level of the stream,
its spec version is the Ion version of the current segment.

```ion
$ion_1_1
$ion_encoding::(
$ion::
(module foo
// Module semantics specified by Ion 1.1
...
)

// ...

$ion_1_3
$ion_encoding::(
$ion::
(module foo
// Module semantics specified by Ion 1.3
...
)
//... // Assuming no IVM
$ion_encoding::(
$ion::
(module bar
// Module semantics specified by Ion 1.3
...
)
Expand Down
53 changes: 26 additions & 27 deletions _books/ion-1-1/src/modules/defining_modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,32 +7,27 @@ A module is defined by four kinds of subclauses which, if present, always appear
3. `symbol_table` - an exported list of text values
4. `macro_table` - an exported list of macro definitions

The lexical name given to a module definition must be an [identifier](../modules.md#identifiers).
However, it must not begin with a `$`--this is reserved for system-defined bindings like `$ion`.

### Internal environment

The body of a module tracks an internal environment by which macro references are resolved.
This environment is constructed incrementally by each clause in the definition and consists of:

* the _visible modules_, a map from identifier to module
* the _module bindings_, a map from identifier to module definition
* the _exported symbols_, an array containing symbol texts
* the _exported macros_, an array containing name/macro pairs

Before any clauses of the module definition are examined, the initial environment is as follows:

* The visible modules map binds `$ion` to the system module for the appropriate spec version.
Inside an encoding directive, the visible modules map also binds `$ion_encoding` to the active encoding module
(the encoding module that was active when the encoding directive was encountered).
For an inner module, it also includes the modules previously made available by the enclosing
module (via `import` or `module`).
* The macro table and symbol table are empty.
Before any clauses of the module definition are examined, each of these is empty.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The modules bindings does not contain $ion or _?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct. The bindings for $ion and _ are at the stream level. Referencing them inside a module will result in recursive lookups through the parent scopes until it reaches the stream. (Optimizations are possible, of course, but this is the behavior that must be achieved.)


Each clause affects the environment as follows:

* An `import` declaration retrieves a shared module from the implementation’s catalog, assigns
it a name in the visible modules, and makes its macros available for use.
An error must be signaled if the name already appears in the visible modules.
* A `module` declaration defines a new module and assigns it a name in the visible modules.
An error must be signaled if the name already appears in the visible modules.
* An `import` declaration retrieves a shared module from the implementation’s catalog and binds a name to it,
making its macros available for use.
An error must be signaled if the name already appears in the module bindings.
* A `module` declaration defines a new module and binds a name to it.
An error must be signaled if the name already appears in the module bindings.
* A `symbol_table` declaration defines the exported symbols.
* A `macro_table` declaration defines the exported macros.

Expand All @@ -53,19 +48,22 @@ macro-addr ::= unannotated-uint

Macro references are resolved to a specific macro as follows:

* An unqualified _macro-name_ is looked up within the exported macros, and if not found, then the
active encoding module's macro table. If it maps to a macro, that’s the resolution of the reference.
Otherwise, an error is signaled due to an unbound reference.
* An anonymous local reference (_macro-addr_) is resolved by index in the exported macro array.
* An unqualified _macro-name_ is looked up in the following locations:
1. in the macros already exported in this module's `macro_table`
2. in the [default_module](encoding_modules.md#the-default-module)
3. in the [system module](system_module.md)

If it maps to a macro, that’s the resolution of the reference. Otherwise, an error is signaled due to an unbound reference.

* An anonymous local reference (_macro-addr_) is resolved by index in the exported macro array.
If the address exceeds the array boundary, an error is signaled due to an invalid reference.
* A qualified reference (_qualified-ref_) resolves solely against the referenced module.
If the module name does not exist in the visible modules, an error is signaled due to an unbound reference.
Otherwise, the name or address is resolved within that module’s exported macro array.
First, the module name must be resolved to a module definition.
* If the module name is in the module bindings, it resolves to the corresponding module definition.
* If the module name is not in the module bindings, resolution is attempted recursively upwards through the parent scopes.
* If the search reaches the top level without resolving to a module, an error is signaled due to an unbound reference.

> [!WARNING]
> An unqualified macro name can change meaning in the middle of an encoding module if you choose to shadow the
> name of a macro in the active encoding module. To unambiguously refer to the active encoding module,
> use the qualified reference syntax: `$ion_encoding::<macro-name>`.
Next, the name or address is resolved within that module definition’s exported macro table.


### `import`
Expand All @@ -82,7 +80,8 @@ catalog-name ::= string
catalog-version ::= int // positive, unannotated
```

An import binds a lexically scoped module name to a shared module that is identified by a catalog key—a `(name, version)` pair. The `version` of the catalog key is optional—when omitted, the version is implicitly 1.
An import binds a lexically scoped module name to a shared module that is identified by a catalog key—a `(name, version)` pair.
The `version` of the catalog key is optional—when omitted, the version is implicitly 1.

In Ion 1.0, imports may be substituted with a different version if an exact match is not found.
In Ion 1.1, however, all imports require an exact match to be found in the reader's catalog;
Expand All @@ -92,7 +91,7 @@ if an exact match is not found, the implementation must signal an error.

### `module`

The `module` clause defines a new module that is contained in the current module.
The `module` clause defines a new local module that is contained in the current module.

```bnf
inner-module ::= '(module' module-name import* symbol-table? macro-table? ')'
Expand All @@ -102,7 +101,7 @@ Inner modules automatically have access to modules previously declared in the co
The new module (and its exported symbols and macros) is available to any following `module`, `symbol_table`, and
`macro_table` clauses in the enclosing container.

See [inner modules](inner_modules.md) for full explanation.
See [local modules](local_modules.md) for full explanation.

### `symbol_table`

Expand Down
81 changes: 81 additions & 0 deletions _books/ion-1-1/src/modules/directives.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Directives

_Directives_ are system values that modify the encoding context.

Syntactically, a directive is a top-level s-expression annotated with `$ion`.
Its first child value is an operation name.
The operation determines what changes will be made to the encoding context and which clauses may legally follow.

```ion
$ion::
(operation_name
(clause_1 /*...*/)
(clause_2 /*...*/)
/*...more clauses...*/
(clause_N /*...*/))
```

In Ion 1.1, there are three supported directive operations:
1. [`module`](#module-directives)
2. [`import`](#import-directives)
3. [`encoding`](#encoding-directives)

## Top-level bindings

The `module` and `import` directives each create a stream-level binding to a module definition.
Once created, module bindings at this level endure until the file ends or another Ion version marker is encountered.

Module bindings at the stream-level can be redefined.

> [!TIP]
> The [`add_macros`](../macros/system_macros.md#add_macros) and [`add_symbols`](../macros/system_macros.md#add_symbols)
> system macros work by redefining the default module (`_`) in terms of itself.

This behavior differs from module bindings created inside another module;
[attempting to redefine these will raise an error](defining_modules.md#internal-environment).

### `module` directives
The `module` directive binds a name to a [local module](local_modules.md) definition at the top level of the stream.

```ion
$ion::
(module foo
/*...imports, if any...*/
/*...submodules, if any...*/
(macro_table /*...*/)
(symbol_table /*...*/)
)
```

### `import` directives

The _import_ directive looks up the module corresponding to the given `(name, version)` pair in the catalog.
Upon success, it creates a new binding to that module at the top level of the stream.

```ion
$ion::
(import
bar // Binding
zslayton marked this conversation as resolved.
Show resolved Hide resolved
"com.example.bar" // Module name
2) // Module version
zslayton marked this conversation as resolved.
Show resolved Hide resolved
```
The `version` can be omitted. When it is not specified, it defaults to `1`.

If the catalog does contain an exact match, this operation raises an error.

## `encoding` directives

An `encoding` directive accepts a sequence of module bindings to use as the following stream segment's
[encoding module sequence](encoding_modules.md).

```ion
$ion::
(encoding
mod_a
mod_b
mod_c)
```

The new encoding module sequence takes effect immediately after the directive and remains the same until the next `encoding` directive or Ion version marker.

Note that the [default module](encoding_modules.md#the-default-module) is always implicitly at the head of the encoding module sequence.
75 changes: 0 additions & 75 deletions _books/ion-1-1/src/modules/encoding_module.md

This file was deleted.

Loading