diff --git a/develop/__pycache__/macro.cpython-38.pyc b/develop/__pycache__/macro.cpython-38.pyc index 19431de44..b71de4166 100644 Binary files a/develop/__pycache__/macro.cpython-38.pyc and b/develop/__pycache__/macro.cpython-38.pyc differ diff --git a/develop/info/index.html b/develop/info/index.html index 214ec77f3..d29922267 100644 --- a/develop/info/index.html +++ b/develop/info/index.html @@ -1063,7 +1063,7 @@

General ListMkDocsConfig - {'config_file_path': '/home/runner/work/spoofax-pie/spoofax-pie/mkdocs.yml', 'site_name': 'Spoofax 3', 'nav': [{'Home': 'index.md'}, {'Tutorials': ['tutorial/install.md', 'tutorial/create_language_project.md', 'tutorial/change_static_semantics.md', 'tutorial/add_transformation.md', 'tutorial/pcf_tutorial.md']}, {'How-to Guides': ['guide/ask_for_help.md', 'guide/report_a_bug.md', {'Eclipse LWB': ['guide/eclipse_lwb/import.md', 'guide/eclipse_lwb/update.md', 'guide/eclipse_lwb/troubleshooting.md']}, {'Static Semantics': ['guide/static-semantics/code-completion.md']}, {'Development': ['guide/development/debugging-in-intellij.md', 'guide/development/troubleshooting.md']}]}, {'Reference': ['reference/configuration.md', 'reference/eclipse-lwb/eclipse-project-files.md']}, {'Background': ['background/documentation.md', 'background/motivation.md', 'background/key_ideas.md', 'background/status.md']}, {'Releases': ['release/download.md']}], 'pages': None, 'site_url': None, 'site_description': 'Spoofax 3 documentation website', 'site_author': None, 'theme': Theme(name='material', dirs=['/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/material', '/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''), language='en', direction=None, features=['navigation.expand', 'navigation.instant', 'navigation.tabs', 'navigation.tabs.sticky', 'navigation.sections', 'navigation.indexes', 'navigation.top'], palette={'primary': 'indigo', 'accent': 'indigo'}, font={'code': 'JetBrains Mono'}, icon=None, favicon='assets/spoofax.png', include_search_page=False, search_index_only=True, logo='assets/spoofax.svg'), 'docs_dir': '/home/runner/work/spoofax-pie/spoofax-pie/docs', 'site_dir': '/home/runner/work/spoofax-pie/spoofax-pie/site', 'copyright': None, 'google_analytics': None, 'dev_addr': _IpAddressValue(host='127.0.0.1', port=8000), 'use_directory_urls': True, 'repo_url': 'https://github.com/metaborg/spoofax-pie', 'repo_name': 'metaborg/spoofax-pie', 'edit_uri_template': None, 'edit_uri': 'edit/master/docs/', 'extra_css': ['extra.css'], 'extra_javascript': [], 'extra_templates': [], 'markdown_extensions': ['toc', 'tables', 'fenced_code', 'abbr', 'admonition', 'attr_list', 'codehilite', 'def_list', 'pymdownx.betterem', 'pymdownx.caret', 'pymdownx.details', 'pymdownx.emoji', 'pymdownx.inlinehilite', 'pymdownx.keys', 'pymdownx.magiclink', 'pymdownx.mark', 'pymdownx.saneheaders', 'pymdownx.smartsymbols', 'pymdownx.snippets', 'pymdownx.superfences', 'pymdownx.tabbed', 'pymdownx.tasklist', 'pymdownx.tilde'], 'mdx_configs': {'codehilite': {'guess_lang': False}, 'toc': {'permalink': True}, 'pymdownx.betterem': {'smart_enable': 'all'}, 'pymdownx.emoji': {'emoji_index': , 'emoji_generator': }, 'pymdownx.tasklist': {'custom_checkbox': True}}, 'strict': False, 'remote_branch': 'gh-pages', 'remote_name': 'origin', 'extra': {}, 'plugins': PluginCollection([('search', ), ('macros', ), ('git-revision-date', )]), 'hooks': {}, 'watch': []} + {'config_file_path': '/home/runner/work/spoofax-pie/spoofax-pie/mkdocs.yml', 'site_name': 'Spoofax 3', 'nav': [{'Home': 'index.md'}, {'Tutorials': ['tutorial/install.md', 'tutorial/create_language_project.md', 'tutorial/change_static_semantics.md', 'tutorial/add_transformation.md', 'tutorial/pcf_tutorial.md']}, {'How-to Guides': ['guide/ask_for_help.md', 'guide/report_a_bug.md', {'Eclipse LWB': ['guide/eclipse_lwb/import.md', 'guide/eclipse_lwb/update.md', 'guide/eclipse_lwb/troubleshooting.md']}, {'Static Semantics': ['guide/static-semantics/code-completion.md']}, {'Development': ['guide/development/debugging-in-intellij.md', 'guide/development/troubleshooting.md']}]}, {'Reference': ['reference/configuration.md', 'reference/eclipse-lwb/eclipse-project-files.md']}, {'Background': ['background/documentation.md', 'background/motivation.md', 'background/key_ideas.md', 'background/status.md']}, {'Releases': ['release/download.md']}], 'pages': None, 'site_url': None, 'site_description': 'Spoofax 3 documentation website', 'site_author': None, 'theme': Theme(name='material', dirs=['/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/material', '/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''), language='en', direction=None, features=['navigation.expand', 'navigation.instant', 'navigation.tabs', 'navigation.tabs.sticky', 'navigation.sections', 'navigation.indexes', 'navigation.top'], palette={'primary': 'indigo', 'accent': 'indigo'}, font={'code': 'JetBrains Mono'}, icon=None, favicon='assets/spoofax.png', include_search_page=False, search_index_only=True, logo='assets/spoofax.svg'), 'docs_dir': '/home/runner/work/spoofax-pie/spoofax-pie/docs', 'site_dir': '/home/runner/work/spoofax-pie/spoofax-pie/site', 'copyright': None, 'google_analytics': None, 'dev_addr': _IpAddressValue(host='127.0.0.1', port=8000), 'use_directory_urls': True, 'repo_url': 'https://github.com/metaborg/spoofax-pie', 'repo_name': 'metaborg/spoofax-pie', 'edit_uri_template': None, 'edit_uri': 'edit/master/docs/', 'extra_css': ['extra.css'], 'extra_javascript': [], 'extra_templates': [], 'markdown_extensions': ['toc', 'tables', 'fenced_code', 'abbr', 'admonition', 'attr_list', 'codehilite', 'def_list', 'pymdownx.betterem', 'pymdownx.caret', 'pymdownx.details', 'pymdownx.emoji', 'pymdownx.inlinehilite', 'pymdownx.keys', 'pymdownx.magiclink', 'pymdownx.mark', 'pymdownx.saneheaders', 'pymdownx.smartsymbols', 'pymdownx.snippets', 'pymdownx.superfences', 'pymdownx.tabbed', 'pymdownx.tasklist', 'pymdownx.tilde'], 'mdx_configs': {'codehilite': {'guess_lang': False}, 'toc': {'permalink': True}, 'pymdownx.betterem': {'smart_enable': 'all'}, 'pymdownx.emoji': {'emoji_index': , 'emoji_generator': }, 'pymdownx.tasklist': {'custom_checkbox': True}}, 'strict': False, 'remote_branch': 'gh-pages', 'remote_name': 'origin', 'extra': {}, 'plugins': PluginCollection([('search', ), ('macros', ), ('git-revision-date', )]), 'hooks': {}, 'watch': []} @@ -1093,7 +1093,7 @@

General Listdict - status = True, date [datetime], short_commit = 'b3b4b94', commit = 'b3b4b9407f7c8a7fd272f82dbe8b7adc13138926', tag = '', author = 'Jeff Smits', author_email = 'j.smits-1@tudelft.nl', committer = 'Jeff Smits', committer_email = 'j.smits-1@tudelft.nl', date_ISO = 'Mon Jul 29 15:07:47 2024 +0200', message = 'Update dagger, add jakarta inject annotation dep', raw = 'commit b3b4b9407f7c8a7fd272f82dbe8b7adc13138926\nAuthor: Jeff Smits \nDate: Mon Jul 29 15:07:47 2024 +0200\n\n Update dagger, add jakarta inject annotation dep', root_dir = '/home/runner/work/spoofax-pie/spoofax-pie' + status = True, date [datetime], short_commit = 'aec1bcb', commit = 'aec1bcbef07cce68c7411506b99132762273a448', tag = '', author = 'Jeff Smits', author_email = 'j.smits-1@tudelft.nl', committer = 'Jeff Smits', committer_email = 'j.smits-1@tudelft.nl', date_ISO = 'Mon Jul 29 16:25:35 2024 +0200', message = 'nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\n\nThis only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects', raw = 'commit aec1bcbef07cce68c7411506b99132762273a448\nAuthor: Jeff Smits \nDate: Mon Jul 29 16:25:32 2024 +0200\n\n nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\n \n This only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects', root_dir = '/home/runner/work/spoofax-pie/spoofax-pie' @@ -1163,7 +1163,7 @@

General ListFiles - + @@ -1577,7 +1577,7 @@

Git Informationdatetime - datetime.datetime(2024, 7, 29, 15, 7, 47, tzinfo=tzoffset(None, 7200)) + datetime.datetime(2024, 7, 29, 16, 25, 35, tzinfo=tzoffset(None, 7200)) @@ -1587,7 +1587,7 @@

Git Informationstr - 'b3b4b94' + 'aec1bcb' @@ -1597,7 +1597,7 @@

Git Informationstr - 'b3b4b9407f7c8a7fd272f82dbe8b7adc13138926' + 'aec1bcbef07cce68c7411506b99132762273a448' @@ -1657,7 +1657,7 @@

Git Informationstr - 'Mon Jul 29 15:07:47 2024 +0200' + 'Mon Jul 29 16:25:35 2024 +0200' @@ -1667,7 +1667,7 @@

Git Informationstr - 'Update dagger, add jakarta inject annotation dep' + 'nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\n\nThis only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects' @@ -1677,7 +1677,7 @@

Git Informationstr - 'commit b3b4b9407f7c8a7fd272f82dbe8b7adc13138926\nAuthor: Jeff Smits \nDate: Mon Jul 29 15:07:47 2024 +0200\n\n Update dagger, add jakarta inject annotation dep' + 'commit aec1bcbef07cce68c7411506b99132762273a448\nAuthor: Jeff Smits \nDate: Mon Jul 29 16:25:32 2024 +0200\n\n nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\n \n This only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects' diff --git a/develop/search/search_index.json b/develop/search/search_index.json index edd18c7d5..b4763aee9 100644 --- a/develop/search/search_index.json +++ b/develop/search/search_index.json @@ -1 +1 @@ -{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Spoofax 3 \u00b6 Spoofax 3 is a modular and incremental textual language workbench running on the JVM : a collection of tools and Java libraries that enable the development of textual languages, embeddable into batch compilers, code editors and IDEs, or custom applications. It is a reimplementation of Spoofax 2 , with the goal of being more modular, flexible, and correctly incremental. This documentation website aims to cover language development with Spoofax 3, development of the language workbench itself (e.g., meta-language development), and development of this documentation. Currently, Spoofax 3 is experimental and still a work-in-progress. Therefore, it does not have a stable API, lacks documentation and test coverage, and has not yet been applied to real-world use cases. If you are looking for a more mature alternative, see Spoofax 2 , which Spoofax 3 is based on. The documentation for Spoofax 3 is split into five parts: Tutorials : hands-on tutorials to getting you started with Spoofax 3. How-to guides : step-by-step guides showing how to implement certain features or how to solve common problems. Reference : technical reference describing the various facets of Spoofax 3. Background : background information and discussion on various key topics. Releases : download links, changelogs, and migration guides.","title":"Home"},{"location":"#spoofax-3","text":"Spoofax 3 is a modular and incremental textual language workbench running on the JVM : a collection of tools and Java libraries that enable the development of textual languages, embeddable into batch compilers, code editors and IDEs, or custom applications. It is a reimplementation of Spoofax 2 , with the goal of being more modular, flexible, and correctly incremental. This documentation website aims to cover language development with Spoofax 3, development of the language workbench itself (e.g., meta-language development), and development of this documentation. Currently, Spoofax 3 is experimental and still a work-in-progress. Therefore, it does not have a stable API, lacks documentation and test coverage, and has not yet been applied to real-world use cases. If you are looking for a more mature alternative, see Spoofax 2 , which Spoofax 3 is based on. The documentation for Spoofax 3 is split into five parts: Tutorials : hands-on tutorials to getting you started with Spoofax 3. How-to guides : step-by-step guides showing how to implement certain features or how to solve common problems. Reference : technical reference describing the various facets of Spoofax 3. Background : background information and discussion on various key topics. Releases : download links, changelogs, and migration guides.","title":"Spoofax 3"},{"location":"info/","text":"Macros Plugin Environment \u00b6 General List \u00b6 All available variables and filters within the macros plugin: Variable Type Content extra dict config MkDocsConfig {'config_file_path': '/home/runner/work/spoofax-pie/spoofax-pie/mkdocs.yml', 'site_name': 'Spoofax 3', 'nav': [{'Home': 'index.md'}, {'Tutorials': ['tutorial/install.md', 'tutorial/create_language_project.md', 'tutorial/change_static_semantics.md', 'tutorial/add_transformation.md', 'tutorial/pcf_tutorial.md']}, {'How-to Guides': ['guide/ask_for_help.md', 'guide/report_a_bug.md', {'Eclipse LWB': ['guide/eclipse_lwb/import.md', 'guide/eclipse_lwb/update.md', 'guide/eclipse_lwb/troubleshooting.md']}, {'Static Semantics': ['guide/static-semantics/code-completion.md']}, {'Development': ['guide/development/debugging-in-intellij.md', 'guide/development/troubleshooting.md']}]}, {'Reference': ['reference/configuration.md', 'reference/eclipse-lwb/eclipse-project-files.md']}, {'Background': ['background/documentation.md', 'background/motivation.md', 'background/key_ideas.md', 'background/status.md']}, {'Releases': ['release/download.md']}], 'pages': None, 'site_url': None, 'site_description': 'Spoofax 3 documentation website', 'site_author': None, 'theme': Theme(name='material', dirs=['/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/material', '/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''), language='en', direction=None, features=['navigation.expand', 'navigation.instant', 'navigation.tabs', 'navigation.tabs.sticky', 'navigation.sections', 'navigation.indexes', 'navigation.top'], palette={'primary': 'indigo', 'accent': 'indigo'}, font={'code': 'JetBrains Mono'}, icon=None, favicon='assets/spoofax.png', include_search_page=False, search_index_only=True, logo='assets/spoofax.svg'), 'docs_dir': '/home/runner/work/spoofax-pie/spoofax-pie/docs', 'site_dir': '/home/runner/work/spoofax-pie/spoofax-pie/site', 'copyright': None, 'google_analytics': None, 'dev_addr': _IpAddressValue(host='127.0.0.1', port=8000), 'use_directory_urls': True, 'repo_url': 'https://github.com/metaborg/spoofax-pie', 'repo_name': 'metaborg/spoofax-pie', 'edit_uri_template': None, 'edit_uri': 'edit/master/docs/', 'extra_css': ['extra.css'], 'extra_javascript': [], 'extra_templates': [], 'markdown_extensions': ['toc', 'tables', 'fenced_code', 'abbr', 'admonition', 'attr_list', 'codehilite', 'def_list', 'pymdownx.betterem', 'pymdownx.caret', 'pymdownx.details', 'pymdownx.emoji', 'pymdownx.inlinehilite', 'pymdownx.keys', 'pymdownx.magiclink', 'pymdownx.mark', 'pymdownx.saneheaders', 'pymdownx.smartsymbols', 'pymdownx.snippets', 'pymdownx.superfences', 'pymdownx.tabbed', 'pymdownx.tasklist', 'pymdownx.tilde'], 'mdx_configs': {'codehilite': {'guess_lang': False}, 'toc': {'permalink': True}, 'pymdownx.betterem': {'smart_enable': 'all'}, 'pymdownx.emoji': {'emoji_index': , 'emoji_generator': }, 'pymdownx.tasklist': {'custom_checkbox': True}}, 'strict': False, 'remote_branch': 'gh-pages', 'remote_name': 'origin', 'extra': {}, 'plugins': PluginCollection([('search', ), ('macros', ), ('git-revision-date', )]), 'hooks': {}, 'watch': []} environment dict system = 'Linux', system_version = '6.5.0-1024-azure', python_version = '3.8.18', mkdocs_version = '1.4.0', macros_plugin_version = '0.7.0', jinja2_version = '3.1.4' plugin LegacyConfig {'module_name': 'docs/macro', 'modules': [], 'include_dir': 'docs/include', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '', 'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False} git dict status = True, date [ datetime ], short_commit = 'b3b4b94', commit = 'b3b4b9407f7c8a7fd272f82dbe8b7adc13138926', tag = '', author = 'Jeff Smits', author_email = 'j.smits-1@tudelft.nl', committer = 'Jeff Smits', committer_email = 'j.smits-1@tudelft.nl', date_ISO = 'Mon Jul 29 15:07:47 2024 +0200', message = 'Update dagger, add jakarta inject annotation dep', raw = 'commit b3b4b9407f7c8a7fd272f82dbe8b7adc13138926\\nAuthor: Jeff Smits \\nDate: Mon Jul 29 15:07:47 2024 +0200\\n\\n Update dagger, add jakarta inject annotation dep', root_dir = '/home/runner/work/spoofax-pie/spoofax-pie' os dict windows = ':fontawesome-brands-windows: Windows', linux = ':fontawesome-brands-linux: Linux', macos = ':fontawesome-brands-apple: macOS' release dict 0.22.0 [ dict ], 0.20.0 [ dict ], 0.19.8 [ dict ], 0.19.7 [ dict ], 0.19.4 [ dict ], 0.19.3 [ dict ], 0.19.2 [ dict ], 0.19.1 [ dict ], 0.19.0 [ dict ], 0.18.0 [ dict ], 0.17.0 [ dict ], 0.16.17 [ dict ], 0.16.16 [ dict ], 0.16.15 [ dict ], 0.16.14 [ dict ], 0.16.13 [ dict ], 0.16.12 [ dict ], 0.16.11 [ dict ], 0.16.10 [ dict ], 0.16.9 [ dict ], 0.16.8 [ dict ], 0.16.7 [ dict ], 0.16.6 [ dict ], 0.16.5 [ dict ], 0.16.4 [ dict ], 0.16.3 [ dict ], 0.16.2 [ dict ], 0.16.1 [ dict ], 0.16.0 [ dict ], 0.15.3 [ dict ], 0.15.2 [ dict ], 0.15.1 [ dict ], 0.15.0 [ dict ], 0.14.2 [ dict ], 0.14.1 [ dict ], 0.14.0 [ dict ], 0.13.0 [ dict ], 0.12.1 [ dict ], 0.12.0 [ dict ], 0.11.13 [ dict ], 0.11.12 [ dict ], 0.11.11 [ dict ], 0.11.10 [ dict ], 0.11.9 [ dict ], 0.11.8 [ dict ], 0.11.7 [ dict ], 0.11.6 [ dict ], 0.11.5 [ dict ], 0.11.4 [ dict ], 0.11.3 [ dict ], 0.11.2 [ dict ], 0.11.1 [ dict ], 0.11.0 [ dict ], 0.10.0 [ dict ], 0.9.0 [ dict ], 0.8.0 [ dict ], rel [ dict ], dev [ dict ] macros SuperDict context [ function ], macros_info [ function ], now [ function ], fix_url [ function ] filters dict pretty [ function ] filters_builtin dict abs [ builtin_function_or_method ], attr [ function ], batch [ function ], capitalize [ function ], center [ function ], count [ builtin_function_or_method ], d [ function ], default [ function ], dictsort [ function ], e [ builtin_function_or_method ], escape [ builtin_function_or_method ], filesizeformat [ function ], first [ function ], float [ function ], forceescape [ function ], format [ function ], groupby [ function ], indent [ function ], int [ function ], join [ function ], last [ function ], length [ builtin_function_or_method ], list [ function ], lower [ function ], items [ function ], map [ function ], min [ function ], max [ function ], pprint [ function ], random [ function ], reject [ function ], rejectattr [ function ], replace [ function ], reverse [ function ], round [ function ], safe [ function ], select [ function ], selectattr [ function ], slice [ function ], sort [ function ], string [ builtin_function_or_method ], striptags [ function ], sum [ function ], title [ function ], trim [ function ], truncate [ function ], unique [ function ], upper [ function ], urlencode [ function ], urlize [ function ], wordcount [ function ], wordwrap [ function ], xmlattr [ function ], tojson [ function ] navigation Navigation Page(title='Home', url='.') Section(title='Tutorials') Page(title=[blank], url='tutorial/install/') Page(title=[blank], url='tutorial/create_language_project/') Page(title=[blank], url='tutorial/change_static_semantics/') Page(title=[blank], url='tutorial/add_transformation/') Page(title=[blank], url='tutorial/pcf_tutorial/') Section(title='How-to Guides') Page(title=[blank], url='guide/ask_for_help/') Page(title=[blank], url='guide/report_a_bug/') Section(title='Eclipse LWB') Page(title=[blank], url='guide/eclipse_lwb/import/') Page(title=[blank], url='guide/eclipse_lwb/update/') Page(title=[blank], url='guide/eclipse_lwb/troubleshooting/') Section(title='Static Semantics') Page(title=[blank], url='guide/static-semantics/code-completion/') Section(title='Development') Page(title=[blank], url='guide/development/debugging-in-intellij/') Page(title=[blank], url='guide/development/troubleshooting/') Section(title='Reference') Page(title=[blank], url='reference/configuration/') Page(title=[blank], url='reference/eclipse-lwb/eclipse-project-files/') Section(title='Background') Page(title=[blank], url='background/documentation/') Page(title=[blank], url='background/motivation/') Page(title=[blank], url='background/key_ideas/') Page(title=[blank], url='background/status/') Section(title='Releases') Page(title=[blank], url='release/download/') files Files page Page Page(title='Info', url='info/') Config Information \u00b6 Standard MkDocs configuration information. Do not try to modify. e.g. {{ config.docs_dir }} See also the MkDocs documentation on the config object . Variable Type Content config_file_path str '/home/runner/work/spoofax-pie/spoofax-pie/mkdocs.yml' site_name str 'Spoofax 3' nav list [{'Home': 'index.md'}, {'Tutorials': ['tutorial/install.md', 'tutorial/create_language_project.md', 'tutorial/change_static_semantics.md', 'tutorial/add_transformation.md', 'tutorial/pcf_tutorial.md']}, {'How-to Guides': ['guide/ask_for_help.md', 'guide/report_a_bug.md', {'Eclipse LWB': ['guide/eclipse_lwb/import.md', 'guide/eclipse_lwb/update.md', 'guide/eclipse_lwb/troubleshooting.md']}, {'Static Semantics': ['guide/static-semantics/code-completion.md']}, {'Development': ['guide/development/debugging-in-intellij.md', 'guide/development/troubleshooting.md']}]}, {'Reference': ['reference/configuration.md', 'reference/eclipse-lwb/eclipse-project-files.md']}, {'Background': ['background/documentation.md', 'background/motivation.md', 'background/key_ideas.md', 'background/status.md']}, {'Releases': ['release/download.md']}] pages NoneType None site_url NoneType None site_description str 'Spoofax 3 documentation website' site_author NoneType None theme Theme Theme(name='material', dirs=['/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/material', '/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''), language='en', direction=None, features=['navigation.expand', 'navigation.instant', 'navigation.tabs', 'navigation.tabs.sticky', 'navigation.sections', 'navigation.indexes', 'navigation.top'], palette={'primary': 'indigo', 'accent': 'indigo'}, font={'code': 'JetBrains Mono'}, icon=None, favicon='assets/spoofax.png', include_search_page=False, search_index_only=True, logo='assets/spoofax.svg') docs_dir str '/home/runner/work/spoofax-pie/spoofax-pie/docs' site_dir str '/home/runner/work/spoofax-pie/spoofax-pie/site' copyright NoneType None google_analytics NoneType None dev_addr _IpAddressValue _IpAddressValue(host='127.0.0.1', port=8000) use_directory_urls bool True repo_url str 'https://github.com/metaborg/spoofax-pie' repo_name str 'metaborg/spoofax-pie' edit_uri_template NoneType None edit_uri str 'edit/master/docs/' extra_css list ['extra.css'] extra_javascript list [] extra_templates list [] markdown_extensions list ['toc', 'tables', 'fenced_code', 'abbr', 'admonition', 'attr_list', 'codehilite', 'def_list', 'pymdownx.betterem', 'pymdownx.caret', 'pymdownx.details', 'pymdownx.emoji', 'pymdownx.inlinehilite', 'pymdownx.keys', 'pymdownx.magiclink', 'pymdownx.mark', 'pymdownx.saneheaders', 'pymdownx.smartsymbols', 'pymdownx.snippets', 'pymdownx.superfences', 'pymdownx.tabbed', 'pymdownx.tasklist', 'pymdownx.tilde'] mdx_configs dict codehilite [ dict ], toc [ dict ], pymdownx.betterem [ dict ], pymdownx.emoji [ dict ], pymdownx.tasklist [ dict ] strict bool False remote_branch str 'gh-pages' remote_name str 'origin' extra LegacyConfig {} plugins PluginCollection search [ SearchPlugin ], macros [ MacrosPlugin ], git-revision-date [ GitRevisionDatePlugin ] hooks dict watch list [] Macros \u00b6 These macros have been defined programmatically for this environment (module or pluglets). Variable Type Content context function ( obj, e ) Default mkdocs_macro List the defined variables macros_info function ( ) Test/debug function: list useful documentation on the mkdocs_macro environment. now function ( ) Get the current time (returns a datetime object). Used alone, it provides a timestamp. To get the year use now().year , for the month number now().month , etc. fix_url function ( url, r ) If url is relative, fix it so that it points to the docs diretory. This is necessary because relative links in markdown must be adapted in html ('img/foo.png' => '../img/img.png'). Git Information \u00b6 Information available on the last commit and the git repository containing the documentation project: e.g. {{ git.message }} Variable Type Content status bool True date datetime datetime.datetime(2024, 7, 29, 15, 7, 47, tzinfo=tzoffset(None, 7200)) short_commit str 'b3b4b94' commit str 'b3b4b9407f7c8a7fd272f82dbe8b7adc13138926' tag str '' author str 'Jeff Smits' author_email str 'j.smits-1@tudelft.nl' committer str 'Jeff Smits' committer_email str 'j.smits-1@tudelft.nl' date_ISO str 'Mon Jul 29 15:07:47 2024 +0200' message str 'Update dagger, add jakarta inject annotation dep' raw str 'commit b3b4b9407f7c8a7fd272f82dbe8b7adc13138926\\nAuthor: Jeff Smits \\nDate: Mon Jul 29 15:07:47 2024 +0200\\n\\n Update dagger, add jakarta inject annotation dep' root_dir str '/home/runner/work/spoofax-pie/spoofax-pie' Page Attributes \u00b6 Provided by MkDocs. These attributes change for every page (the attributes shown are for this page). e.g. {{ page.title }} See also the MkDocs documentation on the page object . Variable Type Content file File page [ Page ], src_uri = 'info.md', abs_src_path = '/home/runner/work/spoofax-pie/spoofax-pie/docs/info.md', name = 'info', dest_uri = 'info/index.html', abs_dest_path = '/home/runner/work/spoofax-pie/spoofax-pie/site/info/index.html', url = 'info/' title str 'Info' parent NoneType None children NoneType None previous_page NoneType None next_page NoneType None _Page__active bool False update_date str '2024-07-29' canonical_url NoneType None abs_url NoneType None edit_url str 'https://github.com/metaborg/spoofax-pie/edit/master/docs/info.md' markdown str '{{ macros_info() }}\\n' content NoneType None toc list [] meta dict To have all titles of all pages, use: {% for page in navigation.pages %} - {{ page.title }} {% endfor %} Plugin Filters \u00b6 These filters are provided as a standard by the macros plugin. Variable Type Content pretty function ( var_list, rows, header, e ) Default mkdocs_macro Prettify a dictionary or object (used for environment documentation, or debugging). Builtin Jinja2 Filters \u00b6 These filters are provided by Jinja2 as a standard. See also the Jinja2 documentation on builtin filters ). Variable Type Content abs builtin_function_or_method Return the absolute value of the argument. attr function ( environment, obj, name, value ) Get an attribute of an object. foo|attr(\"bar\") works like foo.bar just that always an attribute is returned and items are not looked up. batch function ( value, linecount, fill_with, tmp, item ) A filter that batches items. It works pretty much like slice just the other way round. It returns a list of lists with the given number of items. If you provide a second parameter this is used to fill up missing items. See this example. capitalize function ( s ) Capitalize a value. The first character will be uppercase, all others lowercase. center function ( value, width ) Centers the value in a field of a given width. count builtin_function_or_method Return the number of items in a container. d function ( value, default_value, boolean ) If the value is undefined it will return the passed default value, otherwise the value of the variable. default function ( value, default_value, boolean ) If the value is undefined it will return the passed default value, otherwise the value of the variable. dictsort function ( value, case_sensitive, by, reverse, sort_func ) Sort a dict and yield (key, value) pairs. Python dicts may not be in the order you want to display them in, so sort them first. e builtin_function_or_method Replace the characters & , < , > , ' , and \" in the string with HTML-safe sequences. Use this if you need to display text that might contain such characters in HTML. escape builtin_function_or_method Replace the characters & , < , > , ' , and \" in the string with HTML-safe sequences. Use this if you need to display text that might contain such characters in HTML. filesizeformat function ( value, binary, bytes, base, prefixes, i, prefix, unit ) Format the value like a 'human-readable' file size (i.e. 13 kB, 4.1 MB, 102 Bytes, etc). Per default decimal prefixes are used (Mega, Giga, etc.), if the second parameter is set to True the binary prefixes are used (Mebi, Gibi). first function ( args, kwargs, b ) Return the first item of a sequence. float function ( value, default ) Convert the value into a floating point number. If the conversion doesn't work it will return 0.0 . You can override this default using the first parameter. forceescape function ( value ) Enforce HTML escaping. This will probably double escape variables. format function ( value, args, kwargs ) Apply the given values to a printf-style _ format string, like string % values . groupby function ( args, kwargs, b ) Group a sequence of objects by an attribute using Python's :func: itertools.groupby . The attribute can use dot notation for nested access, like \"address.city\" . Unlike Python's groupby , the values are sorted first so only one group is returned for each unique value. indent function ( s, width, first, blank, newline, rv, lines ) Return a copy of the string with each line indented by 4 spaces. The first line and blank lines are not indented by default. int function ( value, default, base ) Convert the value into an integer. If the conversion doesn't work it will return 0 . You can override this default using the first parameter. You can also override the default base (10) in the second parameter, which handles input with prefixes such as 0b, 0o and 0x for bases 2, 8 and 16 respectively. The base is ignored for decimal numbers and non-string values. join function ( args, kwargs, b ) Return a string which is the concatenation of the strings in the sequence. The separator between elements is an empty string per default, you can define it with the optional parameter. last function ( environment, seq ) Return the last item of a sequence. length builtin_function_or_method Return the number of items in a container. list function ( args, kwargs, b ) Convert the value into a list. If it was a string the returned list will be a list of characters. lower function ( s ) Convert a value to lowercase. items function ( value ) Return an iterator over the (key, value) items of a mapping. map function ( args, kwargs, b ) Applies a filter on a sequence of objects or looks up an attribute. This is useful when dealing with lists of objects but you are really only interested in a certain value of it. min function ( environment, value, case_sensitive, attribute ) Return the smallest item from the sequence. max function ( environment, value, case_sensitive, attribute ) Return the largest item from the sequence. pprint function ( value ) Pretty print a variable. Useful for debugging. random function ( context, seq ) Return a random item from the sequence. reject function ( args, kwargs, b ) Filters a sequence of objects by applying a test to each object, and rejecting the objects with the test succeeding. rejectattr function ( args, kwargs, b ) Filters a sequence of objects by applying a test to the specified attribute of each object, and rejecting the objects with the test succeeding. replace function ( eval_ctx, s, old, new, count ) Return a copy of the value with all occurrences of a substring replaced with a new one. The first argument is the substring that should be replaced, the second is the replacement string. If the optional third argument count is given, only the first count occurrences are replaced. reverse function ( value, rv, e ) Reverse the object or return an iterator that iterates over it the other way round. round function ( value, precision, method, func ) Round the number to a given precision. The first parameter specifies the precision (default is 0 ), the second the rounding method. safe function ( value ) Mark the value as safe which means that in an environment with automatic escaping enabled this variable will not be escaped. select function ( args, kwargs, b ) Filters a sequence of objects by applying a test to each object, and only selecting the objects with the test succeeding. selectattr function ( args, kwargs, b ) Filters a sequence of objects by applying a test to the specified attribute of each object, and only selecting the objects with the test succeeding. slice function ( args, kwargs, b ) Slice an iterator and return a list of lists containing those items. Useful if you want to create a div containing three ul tags that represent columns. sort function ( environment, value, reverse, case_sensitive, attribute, key_func ) Sort an iterable using Python's :func: sorted . string builtin_function_or_method Convert an object to a string if it isn't already. This preserves a :class: Markup string rather than converting it back to a basic string, so it will still be marked as safe and won't be escaped again. striptags function ( value ) Strip SGML/XML tags and replace adjacent whitespace by one space. sum function ( args, kwargs, b ) Returns the sum of a sequence of numbers plus the value of parameter 'start' (which defaults to 0). When the sequence is empty it returns start. title function ( s ) Return a titlecased version of the value. I.e. words will start with uppercase letters, all remaining characters are lowercase. trim function ( value, chars ) Strip leading and trailing characters, by default whitespace. truncate function ( env, s, length, killwords, end, leeway, result ) Return a truncated copy of the string. The length is specified with the first parameter which defaults to 255 . If the second parameter is true the filter will cut the text at length. Otherwise it will discard the last word. If the text was in fact truncated it will append an ellipsis sign ( \"...\" ). If you want a different ellipsis sign than \"...\" you can specify it using the third parameter. Strings that only exceed the length by the tolerance margin given in the fourth parameter will not be truncated. unique function ( environment, value, case_sensitive, attribute, getter, seen, item, key ) Returns a list of unique items from the given iterable. upper function ( s ) Convert a value to uppercase. urlencode function ( value, items ) Quote data for use in a URL path or query using UTF-8. urlize function ( eval_ctx, value, trim_url_limit, nofollow, target, rel, extra_schemes, policies, rel_parts, scheme, rv ) Convert URLs in text into clickable links. wordcount function ( s ) Count the words in that string. wordwrap function ( environment, s, width, break_long_words, wrapstring, break_on_hyphens ) Wrap a string to the given width. Existing newlines are treated as paragraphs to be wrapped separately. xmlattr function ( eval_ctx, d, autospace, items, key, value, rv ) Create an SGML/XML attribute string based on the items in a dict. tojson function ( eval_ctx, value, indent, policies, dumps, kwargs ) Serialize an object to a string of JSON, and mark it safe to render in HTML. This filter is only for use in HTML documents.","title":"Info"},{"location":"info/#macros-plugin-environment","text":"","title":"Macros Plugin Environment"},{"location":"info/#general-list","text":"All available variables and filters within the macros plugin: Variable Type Content extra dict config MkDocsConfig {'config_file_path': '/home/runner/work/spoofax-pie/spoofax-pie/mkdocs.yml', 'site_name': 'Spoofax 3', 'nav': [{'Home': 'index.md'}, {'Tutorials': ['tutorial/install.md', 'tutorial/create_language_project.md', 'tutorial/change_static_semantics.md', 'tutorial/add_transformation.md', 'tutorial/pcf_tutorial.md']}, {'How-to Guides': ['guide/ask_for_help.md', 'guide/report_a_bug.md', {'Eclipse LWB': ['guide/eclipse_lwb/import.md', 'guide/eclipse_lwb/update.md', 'guide/eclipse_lwb/troubleshooting.md']}, {'Static Semantics': ['guide/static-semantics/code-completion.md']}, {'Development': ['guide/development/debugging-in-intellij.md', 'guide/development/troubleshooting.md']}]}, {'Reference': ['reference/configuration.md', 'reference/eclipse-lwb/eclipse-project-files.md']}, {'Background': ['background/documentation.md', 'background/motivation.md', 'background/key_ideas.md', 'background/status.md']}, {'Releases': ['release/download.md']}], 'pages': None, 'site_url': None, 'site_description': 'Spoofax 3 documentation website', 'site_author': None, 'theme': Theme(name='material', dirs=['/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/material', '/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''), language='en', direction=None, features=['navigation.expand', 'navigation.instant', 'navigation.tabs', 'navigation.tabs.sticky', 'navigation.sections', 'navigation.indexes', 'navigation.top'], palette={'primary': 'indigo', 'accent': 'indigo'}, font={'code': 'JetBrains Mono'}, icon=None, favicon='assets/spoofax.png', include_search_page=False, search_index_only=True, logo='assets/spoofax.svg'), 'docs_dir': '/home/runner/work/spoofax-pie/spoofax-pie/docs', 'site_dir': '/home/runner/work/spoofax-pie/spoofax-pie/site', 'copyright': None, 'google_analytics': None, 'dev_addr': _IpAddressValue(host='127.0.0.1', port=8000), 'use_directory_urls': True, 'repo_url': 'https://github.com/metaborg/spoofax-pie', 'repo_name': 'metaborg/spoofax-pie', 'edit_uri_template': None, 'edit_uri': 'edit/master/docs/', 'extra_css': ['extra.css'], 'extra_javascript': [], 'extra_templates': [], 'markdown_extensions': ['toc', 'tables', 'fenced_code', 'abbr', 'admonition', 'attr_list', 'codehilite', 'def_list', 'pymdownx.betterem', 'pymdownx.caret', 'pymdownx.details', 'pymdownx.emoji', 'pymdownx.inlinehilite', 'pymdownx.keys', 'pymdownx.magiclink', 'pymdownx.mark', 'pymdownx.saneheaders', 'pymdownx.smartsymbols', 'pymdownx.snippets', 'pymdownx.superfences', 'pymdownx.tabbed', 'pymdownx.tasklist', 'pymdownx.tilde'], 'mdx_configs': {'codehilite': {'guess_lang': False}, 'toc': {'permalink': True}, 'pymdownx.betterem': {'smart_enable': 'all'}, 'pymdownx.emoji': {'emoji_index': , 'emoji_generator': }, 'pymdownx.tasklist': {'custom_checkbox': True}}, 'strict': False, 'remote_branch': 'gh-pages', 'remote_name': 'origin', 'extra': {}, 'plugins': PluginCollection([('search', ), ('macros', ), ('git-revision-date', )]), 'hooks': {}, 'watch': []} environment dict system = 'Linux', system_version = '6.5.0-1024-azure', python_version = '3.8.18', mkdocs_version = '1.4.0', macros_plugin_version = '0.7.0', jinja2_version = '3.1.4' plugin LegacyConfig {'module_name': 'docs/macro', 'modules': [], 'include_dir': 'docs/include', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '', 'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False} git dict status = True, date [ datetime ], short_commit = 'b3b4b94', commit = 'b3b4b9407f7c8a7fd272f82dbe8b7adc13138926', tag = '', author = 'Jeff Smits', author_email = 'j.smits-1@tudelft.nl', committer = 'Jeff Smits', committer_email = 'j.smits-1@tudelft.nl', date_ISO = 'Mon Jul 29 15:07:47 2024 +0200', message = 'Update dagger, add jakarta inject annotation dep', raw = 'commit b3b4b9407f7c8a7fd272f82dbe8b7adc13138926\\nAuthor: Jeff Smits \\nDate: Mon Jul 29 15:07:47 2024 +0200\\n\\n Update dagger, add jakarta inject annotation dep', root_dir = '/home/runner/work/spoofax-pie/spoofax-pie' os dict windows = ':fontawesome-brands-windows: Windows', linux = ':fontawesome-brands-linux: Linux', macos = ':fontawesome-brands-apple: macOS' release dict 0.22.0 [ dict ], 0.20.0 [ dict ], 0.19.8 [ dict ], 0.19.7 [ dict ], 0.19.4 [ dict ], 0.19.3 [ dict ], 0.19.2 [ dict ], 0.19.1 [ dict ], 0.19.0 [ dict ], 0.18.0 [ dict ], 0.17.0 [ dict ], 0.16.17 [ dict ], 0.16.16 [ dict ], 0.16.15 [ dict ], 0.16.14 [ dict ], 0.16.13 [ dict ], 0.16.12 [ dict ], 0.16.11 [ dict ], 0.16.10 [ dict ], 0.16.9 [ dict ], 0.16.8 [ dict ], 0.16.7 [ dict ], 0.16.6 [ dict ], 0.16.5 [ dict ], 0.16.4 [ dict ], 0.16.3 [ dict ], 0.16.2 [ dict ], 0.16.1 [ dict ], 0.16.0 [ dict ], 0.15.3 [ dict ], 0.15.2 [ dict ], 0.15.1 [ dict ], 0.15.0 [ dict ], 0.14.2 [ dict ], 0.14.1 [ dict ], 0.14.0 [ dict ], 0.13.0 [ dict ], 0.12.1 [ dict ], 0.12.0 [ dict ], 0.11.13 [ dict ], 0.11.12 [ dict ], 0.11.11 [ dict ], 0.11.10 [ dict ], 0.11.9 [ dict ], 0.11.8 [ dict ], 0.11.7 [ dict ], 0.11.6 [ dict ], 0.11.5 [ dict ], 0.11.4 [ dict ], 0.11.3 [ dict ], 0.11.2 [ dict ], 0.11.1 [ dict ], 0.11.0 [ dict ], 0.10.0 [ dict ], 0.9.0 [ dict ], 0.8.0 [ dict ], rel [ dict ], dev [ dict ] macros SuperDict context [ function ], macros_info [ function ], now [ function ], fix_url [ function ] filters dict pretty [ function ] filters_builtin dict abs [ builtin_function_or_method ], attr [ function ], batch [ function ], capitalize [ function ], center [ function ], count [ builtin_function_or_method ], d [ function ], default [ function ], dictsort [ function ], e [ builtin_function_or_method ], escape [ builtin_function_or_method ], filesizeformat [ function ], first [ function ], float [ function ], forceescape [ function ], format [ function ], groupby [ function ], indent [ function ], int [ function ], join [ function ], last [ function ], length [ builtin_function_or_method ], list [ function ], lower [ function ], items [ function ], map [ function ], min [ function ], max [ function ], pprint [ function ], random [ function ], reject [ function ], rejectattr [ function ], replace [ function ], reverse [ function ], round [ function ], safe [ function ], select [ function ], selectattr [ function ], slice [ function ], sort [ function ], string [ builtin_function_or_method ], striptags [ function ], sum [ function ], title [ function ], trim [ function ], truncate [ function ], unique [ function ], upper [ function ], urlencode [ function ], urlize [ function ], wordcount [ function ], wordwrap [ function ], xmlattr [ function ], tojson [ function ] navigation Navigation Page(title='Home', url='.') Section(title='Tutorials') Page(title=[blank], url='tutorial/install/') Page(title=[blank], url='tutorial/create_language_project/') Page(title=[blank], url='tutorial/change_static_semantics/') Page(title=[blank], url='tutorial/add_transformation/') Page(title=[blank], url='tutorial/pcf_tutorial/') Section(title='How-to Guides') Page(title=[blank], url='guide/ask_for_help/') Page(title=[blank], url='guide/report_a_bug/') Section(title='Eclipse LWB') Page(title=[blank], url='guide/eclipse_lwb/import/') Page(title=[blank], url='guide/eclipse_lwb/update/') Page(title=[blank], url='guide/eclipse_lwb/troubleshooting/') Section(title='Static Semantics') Page(title=[blank], url='guide/static-semantics/code-completion/') Section(title='Development') Page(title=[blank], url='guide/development/debugging-in-intellij/') Page(title=[blank], url='guide/development/troubleshooting/') Section(title='Reference') Page(title=[blank], url='reference/configuration/') Page(title=[blank], url='reference/eclipse-lwb/eclipse-project-files/') Section(title='Background') Page(title=[blank], url='background/documentation/') Page(title=[blank], url='background/motivation/') Page(title=[blank], url='background/key_ideas/') Page(title=[blank], url='background/status/') Section(title='Releases') Page(title=[blank], url='release/download/') files Files page Page Page(title='Info', url='info/')","title":"General List"},{"location":"info/#config-information","text":"Standard MkDocs configuration information. Do not try to modify. e.g. {{ config.docs_dir }} See also the MkDocs documentation on the config object . Variable Type Content config_file_path str '/home/runner/work/spoofax-pie/spoofax-pie/mkdocs.yml' site_name str 'Spoofax 3' nav list [{'Home': 'index.md'}, {'Tutorials': ['tutorial/install.md', 'tutorial/create_language_project.md', 'tutorial/change_static_semantics.md', 'tutorial/add_transformation.md', 'tutorial/pcf_tutorial.md']}, {'How-to Guides': ['guide/ask_for_help.md', 'guide/report_a_bug.md', {'Eclipse LWB': ['guide/eclipse_lwb/import.md', 'guide/eclipse_lwb/update.md', 'guide/eclipse_lwb/troubleshooting.md']}, {'Static Semantics': ['guide/static-semantics/code-completion.md']}, {'Development': ['guide/development/debugging-in-intellij.md', 'guide/development/troubleshooting.md']}]}, {'Reference': ['reference/configuration.md', 'reference/eclipse-lwb/eclipse-project-files.md']}, {'Background': ['background/documentation.md', 'background/motivation.md', 'background/key_ideas.md', 'background/status.md']}, {'Releases': ['release/download.md']}] pages NoneType None site_url NoneType None site_description str 'Spoofax 3 documentation website' site_author NoneType None theme Theme Theme(name='material', dirs=['/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/material', '/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''), language='en', direction=None, features=['navigation.expand', 'navigation.instant', 'navigation.tabs', 'navigation.tabs.sticky', 'navigation.sections', 'navigation.indexes', 'navigation.top'], palette={'primary': 'indigo', 'accent': 'indigo'}, font={'code': 'JetBrains Mono'}, icon=None, favicon='assets/spoofax.png', include_search_page=False, search_index_only=True, logo='assets/spoofax.svg') docs_dir str '/home/runner/work/spoofax-pie/spoofax-pie/docs' site_dir str '/home/runner/work/spoofax-pie/spoofax-pie/site' copyright NoneType None google_analytics NoneType None dev_addr _IpAddressValue _IpAddressValue(host='127.0.0.1', port=8000) use_directory_urls bool True repo_url str 'https://github.com/metaborg/spoofax-pie' repo_name str 'metaborg/spoofax-pie' edit_uri_template NoneType None edit_uri str 'edit/master/docs/' extra_css list ['extra.css'] extra_javascript list [] extra_templates list [] markdown_extensions list ['toc', 'tables', 'fenced_code', 'abbr', 'admonition', 'attr_list', 'codehilite', 'def_list', 'pymdownx.betterem', 'pymdownx.caret', 'pymdownx.details', 'pymdownx.emoji', 'pymdownx.inlinehilite', 'pymdownx.keys', 'pymdownx.magiclink', 'pymdownx.mark', 'pymdownx.saneheaders', 'pymdownx.smartsymbols', 'pymdownx.snippets', 'pymdownx.superfences', 'pymdownx.tabbed', 'pymdownx.tasklist', 'pymdownx.tilde'] mdx_configs dict codehilite [ dict ], toc [ dict ], pymdownx.betterem [ dict ], pymdownx.emoji [ dict ], pymdownx.tasklist [ dict ] strict bool False remote_branch str 'gh-pages' remote_name str 'origin' extra LegacyConfig {} plugins PluginCollection search [ SearchPlugin ], macros [ MacrosPlugin ], git-revision-date [ GitRevisionDatePlugin ] hooks dict watch list []","title":"Config Information"},{"location":"info/#macros","text":"These macros have been defined programmatically for this environment (module or pluglets). Variable Type Content context function ( obj, e ) Default mkdocs_macro List the defined variables macros_info function ( ) Test/debug function: list useful documentation on the mkdocs_macro environment. now function ( ) Get the current time (returns a datetime object). Used alone, it provides a timestamp. To get the year use now().year , for the month number now().month , etc. fix_url function ( url, r ) If url is relative, fix it so that it points to the docs diretory. This is necessary because relative links in markdown must be adapted in html ('img/foo.png' => '../img/img.png').","title":"Macros"},{"location":"info/#git-information","text":"Information available on the last commit and the git repository containing the documentation project: e.g. {{ git.message }} Variable Type Content status bool True date datetime datetime.datetime(2024, 7, 29, 15, 7, 47, tzinfo=tzoffset(None, 7200)) short_commit str 'b3b4b94' commit str 'b3b4b9407f7c8a7fd272f82dbe8b7adc13138926' tag str '' author str 'Jeff Smits' author_email str 'j.smits-1@tudelft.nl' committer str 'Jeff Smits' committer_email str 'j.smits-1@tudelft.nl' date_ISO str 'Mon Jul 29 15:07:47 2024 +0200' message str 'Update dagger, add jakarta inject annotation dep' raw str 'commit b3b4b9407f7c8a7fd272f82dbe8b7adc13138926\\nAuthor: Jeff Smits \\nDate: Mon Jul 29 15:07:47 2024 +0200\\n\\n Update dagger, add jakarta inject annotation dep' root_dir str '/home/runner/work/spoofax-pie/spoofax-pie'","title":"Git Information"},{"location":"info/#page-attributes","text":"Provided by MkDocs. These attributes change for every page (the attributes shown are for this page). e.g. {{ page.title }} See also the MkDocs documentation on the page object . Variable Type Content file File page [ Page ], src_uri = 'info.md', abs_src_path = '/home/runner/work/spoofax-pie/spoofax-pie/docs/info.md', name = 'info', dest_uri = 'info/index.html', abs_dest_path = '/home/runner/work/spoofax-pie/spoofax-pie/site/info/index.html', url = 'info/' title str 'Info' parent NoneType None children NoneType None previous_page NoneType None next_page NoneType None _Page__active bool False update_date str '2024-07-29' canonical_url NoneType None abs_url NoneType None edit_url str 'https://github.com/metaborg/spoofax-pie/edit/master/docs/info.md' markdown str '{{ macros_info() }}\\n' content NoneType None toc list [] meta dict To have all titles of all pages, use: {% for page in navigation.pages %} - {{ page.title }} {% endfor %}","title":"Page Attributes"},{"location":"info/#plugin-filters","text":"These filters are provided as a standard by the macros plugin. Variable Type Content pretty function ( var_list, rows, header, e ) Default mkdocs_macro Prettify a dictionary or object (used for environment documentation, or debugging).","title":"Plugin Filters"},{"location":"info/#builtin-jinja2-filters","text":"These filters are provided by Jinja2 as a standard. See also the Jinja2 documentation on builtin filters ). Variable Type Content abs builtin_function_or_method Return the absolute value of the argument. attr function ( environment, obj, name, value ) Get an attribute of an object. foo|attr(\"bar\") works like foo.bar just that always an attribute is returned and items are not looked up. batch function ( value, linecount, fill_with, tmp, item ) A filter that batches items. It works pretty much like slice just the other way round. It returns a list of lists with the given number of items. If you provide a second parameter this is used to fill up missing items. See this example. capitalize function ( s ) Capitalize a value. The first character will be uppercase, all others lowercase. center function ( value, width ) Centers the value in a field of a given width. count builtin_function_or_method Return the number of items in a container. d function ( value, default_value, boolean ) If the value is undefined it will return the passed default value, otherwise the value of the variable. default function ( value, default_value, boolean ) If the value is undefined it will return the passed default value, otherwise the value of the variable. dictsort function ( value, case_sensitive, by, reverse, sort_func ) Sort a dict and yield (key, value) pairs. Python dicts may not be in the order you want to display them in, so sort them first. e builtin_function_or_method Replace the characters & , < , > , ' , and \" in the string with HTML-safe sequences. Use this if you need to display text that might contain such characters in HTML. escape builtin_function_or_method Replace the characters & , < , > , ' , and \" in the string with HTML-safe sequences. Use this if you need to display text that might contain such characters in HTML. filesizeformat function ( value, binary, bytes, base, prefixes, i, prefix, unit ) Format the value like a 'human-readable' file size (i.e. 13 kB, 4.1 MB, 102 Bytes, etc). Per default decimal prefixes are used (Mega, Giga, etc.), if the second parameter is set to True the binary prefixes are used (Mebi, Gibi). first function ( args, kwargs, b ) Return the first item of a sequence. float function ( value, default ) Convert the value into a floating point number. If the conversion doesn't work it will return 0.0 . You can override this default using the first parameter. forceescape function ( value ) Enforce HTML escaping. This will probably double escape variables. format function ( value, args, kwargs ) Apply the given values to a printf-style _ format string, like string % values . groupby function ( args, kwargs, b ) Group a sequence of objects by an attribute using Python's :func: itertools.groupby . The attribute can use dot notation for nested access, like \"address.city\" . Unlike Python's groupby , the values are sorted first so only one group is returned for each unique value. indent function ( s, width, first, blank, newline, rv, lines ) Return a copy of the string with each line indented by 4 spaces. The first line and blank lines are not indented by default. int function ( value, default, base ) Convert the value into an integer. If the conversion doesn't work it will return 0 . You can override this default using the first parameter. You can also override the default base (10) in the second parameter, which handles input with prefixes such as 0b, 0o and 0x for bases 2, 8 and 16 respectively. The base is ignored for decimal numbers and non-string values. join function ( args, kwargs, b ) Return a string which is the concatenation of the strings in the sequence. The separator between elements is an empty string per default, you can define it with the optional parameter. last function ( environment, seq ) Return the last item of a sequence. length builtin_function_or_method Return the number of items in a container. list function ( args, kwargs, b ) Convert the value into a list. If it was a string the returned list will be a list of characters. lower function ( s ) Convert a value to lowercase. items function ( value ) Return an iterator over the (key, value) items of a mapping. map function ( args, kwargs, b ) Applies a filter on a sequence of objects or looks up an attribute. This is useful when dealing with lists of objects but you are really only interested in a certain value of it. min function ( environment, value, case_sensitive, attribute ) Return the smallest item from the sequence. max function ( environment, value, case_sensitive, attribute ) Return the largest item from the sequence. pprint function ( value ) Pretty print a variable. Useful for debugging. random function ( context, seq ) Return a random item from the sequence. reject function ( args, kwargs, b ) Filters a sequence of objects by applying a test to each object, and rejecting the objects with the test succeeding. rejectattr function ( args, kwargs, b ) Filters a sequence of objects by applying a test to the specified attribute of each object, and rejecting the objects with the test succeeding. replace function ( eval_ctx, s, old, new, count ) Return a copy of the value with all occurrences of a substring replaced with a new one. The first argument is the substring that should be replaced, the second is the replacement string. If the optional third argument count is given, only the first count occurrences are replaced. reverse function ( value, rv, e ) Reverse the object or return an iterator that iterates over it the other way round. round function ( value, precision, method, func ) Round the number to a given precision. The first parameter specifies the precision (default is 0 ), the second the rounding method. safe function ( value ) Mark the value as safe which means that in an environment with automatic escaping enabled this variable will not be escaped. select function ( args, kwargs, b ) Filters a sequence of objects by applying a test to each object, and only selecting the objects with the test succeeding. selectattr function ( args, kwargs, b ) Filters a sequence of objects by applying a test to the specified attribute of each object, and only selecting the objects with the test succeeding. slice function ( args, kwargs, b ) Slice an iterator and return a list of lists containing those items. Useful if you want to create a div containing three ul tags that represent columns. sort function ( environment, value, reverse, case_sensitive, attribute, key_func ) Sort an iterable using Python's :func: sorted . string builtin_function_or_method Convert an object to a string if it isn't already. This preserves a :class: Markup string rather than converting it back to a basic string, so it will still be marked as safe and won't be escaped again. striptags function ( value ) Strip SGML/XML tags and replace adjacent whitespace by one space. sum function ( args, kwargs, b ) Returns the sum of a sequence of numbers plus the value of parameter 'start' (which defaults to 0). When the sequence is empty it returns start. title function ( s ) Return a titlecased version of the value. I.e. words will start with uppercase letters, all remaining characters are lowercase. trim function ( value, chars ) Strip leading and trailing characters, by default whitespace. truncate function ( env, s, length, killwords, end, leeway, result ) Return a truncated copy of the string. The length is specified with the first parameter which defaults to 255 . If the second parameter is true the filter will cut the text at length. Otherwise it will discard the last word. If the text was in fact truncated it will append an ellipsis sign ( \"...\" ). If you want a different ellipsis sign than \"...\" you can specify it using the third parameter. Strings that only exceed the length by the tolerance margin given in the fourth parameter will not be truncated. unique function ( environment, value, case_sensitive, attribute, getter, seen, item, key ) Returns a list of unique items from the given iterable. upper function ( s ) Convert a value to uppercase. urlencode function ( value, items ) Quote data for use in a URL path or query using UTF-8. urlize function ( eval_ctx, value, trim_url_limit, nofollow, target, rel, extra_schemes, policies, rel_parts, scheme, rv ) Convert URLs in text into clickable links. wordcount function ( s ) Count the words in that string. wordwrap function ( environment, s, width, break_long_words, wrapstring, break_on_hyphens ) Wrap a string to the given width. Existing newlines are treated as paragraphs to be wrapped separately. xmlattr function ( eval_ctx, d, autospace, items, key, value, rv ) Create an SGML/XML attribute string based on the items in a dict. tojson function ( eval_ctx, value, indent, policies, dumps, kwargs ) Serialize an object to a string of JSON, and mark it safe to render in HTML. This filter is only for use in HTML documents.","title":"Builtin Jinja2 Filters"},{"location":"_include/_all/","text":"","title":" all"},{"location":"_include/abbreviation/","text":"","title":"Abbreviation"},{"location":"background/documentation/","text":"Documentation \u00b6 In this section, we explain the documentation technology and how it is structured. Technology \u00b6 This documentation is developed with MkDocs , a fast and simple static site generated that's geared towards building project documentation from Markdown files. MkDocs uses Python-Markdown to process Markdown files, along with PyMdown Extensions . We use the Material for MkDocs theme which provides a clean look, easy customization, and many features for technical documentation. We use the following MkDocs plugins: mkdocs-macros-plugin to enable the use of variables, macros, and filters in Markdown files. mkdocs-git-revision-date-plugin to add a changed date to the footer based on the last time the file was changed in the Git repository. The documentation is automatically built and published on a commit to the master branch of this repository using the GitHub actions workflow at .github/workflows/documentation.yml . Structure \u00b6 The structure of this documentation follows The documentation system where documentation is split into four categories: Tutorials : oriented to learning , enabling newcomers to get started through a lesson , analogous to teaching a child how to cook . How-to guides : oriented to a particular goal , showing how to solve a specific problem through a series of steps , analogous to a recipe in a cookery book . Reference : oriented to information , describing the machinery through dry description , analogous to an encyclopaedia article . Explanation : oriented to understanding , explaining through discursive explanation , analogous to an article on culinary social history .","title":"Documentation"},{"location":"background/documentation/#documentation","text":"In this section, we explain the documentation technology and how it is structured.","title":"Documentation"},{"location":"background/documentation/#technology","text":"This documentation is developed with MkDocs , a fast and simple static site generated that's geared towards building project documentation from Markdown files. MkDocs uses Python-Markdown to process Markdown files, along with PyMdown Extensions . We use the Material for MkDocs theme which provides a clean look, easy customization, and many features for technical documentation. We use the following MkDocs plugins: mkdocs-macros-plugin to enable the use of variables, macros, and filters in Markdown files. mkdocs-git-revision-date-plugin to add a changed date to the footer based on the last time the file was changed in the Git repository. The documentation is automatically built and published on a commit to the master branch of this repository using the GitHub actions workflow at .github/workflows/documentation.yml .","title":"Technology"},{"location":"background/documentation/#structure","text":"The structure of this documentation follows The documentation system where documentation is split into four categories: Tutorials : oriented to learning , enabling newcomers to get started through a lesson , analogous to teaching a child how to cook . How-to guides : oriented to a particular goal , showing how to solve a specific problem through a series of steps , analogous to a recipe in a cookery book . Reference : oriented to information , describing the machinery through dry description , analogous to an encyclopaedia article . Explanation : oriented to understanding , explaining through discursive explanation , analogous to an article on culinary social history .","title":"Structure"},{"location":"background/key_ideas/","text":"Key ideas \u00b6 To solve the problems highlighted in the motivation section, we intend to employ the following key ideas in Spoofax 3. To reduce coupling, Spoofax 3's \"Spoofax Core\" does not depend on any meta-components. Instead, a language implementation depends directly on the meta-components that it requires. For example, the Tiger language implementation depends directly on the JSGLR2 parser, the NaBL2 constraint solver, and the Stratego runtime. To make language pipelines flexible, modular, and incremental, we use an incremental, modular, and expressive build system as the basis for creating pipelines: PIE . Language processing steps such as parsing, styling text, analyzing, checking (to provide inline error messages), running (parts of) a compiler, etc. become PIE task definitions. Tasks, which are instances of these task definitions, can depend on each other, and depend on resources such as files. The PIE runtime efficiently and incrementally executes tasks. Furthermore, task definitions can be shared and used by other language implementations, making language implementations modular. To reduce the tedium of dynamic language loading, we instead choose to do static language loading as the default. A language implementation is just a JAR file that can be put on the classpath and used as a regular Java library. For example, to use the JSGLR2 parser of the Tiger language, we just depend on the Tiger language implementation as we would depend on a regular Java library, create an instance of the TigerParser class, and then use that to parse a string into an AST. We still want to automatically provide integrations with the command-line, build systems such as Gradle, and IDEs such as Eclipse and IntelliJ. Therefore, every language implementation must implement the LanguageInstance interface. Spoofax 3 then provides libraries which take a LanguageInstance object, and integrate it with a platform. For example, spoofax.cli takes a LanguageInstance object and provides a command-line application, and spoofax.eclipse does the same for an Eclipse plugin. Because language implementations are just regular Java libraries, they now require some Java boilerplate. However, we do not want language developers to write this Java boilerplate for standard cases. Therefore, we employ a Spoofax 3 compiler that generates this Java boilerplate. If the language developer is not happy with the implementation, or wants to customize parts, they can manually implement or extend Java classes where needed. It is also possible to not use the Spoofax 3 compiler at all, and manually implement all parts. To enable quick language prototyping, we still support dynamic language loading in environments that support them (e.g., Eclipse and IntelliJ), by dynamically loading the language implementation JAR when changed. For example, when prototyping the Tiger language in Eclipse, if the syntax definition is changed we run the Spoofax 3 compiler to (incrementally) create a new parse table and Java classes, and dynamically (re)load the JAR. To improve the user experience, we use a configuration DSL to configure language specifications and implementations. Thereby configuration is centralized, has domain-specific checking, and editor services such as inline errors and code completion. We also allow changing of defaults (conventions), and persist them to enable renaming. To improve error traceability, errors are reported inline where possible. Errors are traced through PIE pipelines and support origin tracking to easily support error traceability and inline errors for all language implementations. TODO: better builders: non-Stratego commands incremental commands separate commands from how they are executed support command parameters/arguments continuous execution TODO: modular and incremental development of Spoofax 3 itself with Gradle","title":"Key ideas"},{"location":"background/key_ideas/#key-ideas","text":"To solve the problems highlighted in the motivation section, we intend to employ the following key ideas in Spoofax 3. To reduce coupling, Spoofax 3's \"Spoofax Core\" does not depend on any meta-components. Instead, a language implementation depends directly on the meta-components that it requires. For example, the Tiger language implementation depends directly on the JSGLR2 parser, the NaBL2 constraint solver, and the Stratego runtime. To make language pipelines flexible, modular, and incremental, we use an incremental, modular, and expressive build system as the basis for creating pipelines: PIE . Language processing steps such as parsing, styling text, analyzing, checking (to provide inline error messages), running (parts of) a compiler, etc. become PIE task definitions. Tasks, which are instances of these task definitions, can depend on each other, and depend on resources such as files. The PIE runtime efficiently and incrementally executes tasks. Furthermore, task definitions can be shared and used by other language implementations, making language implementations modular. To reduce the tedium of dynamic language loading, we instead choose to do static language loading as the default. A language implementation is just a JAR file that can be put on the classpath and used as a regular Java library. For example, to use the JSGLR2 parser of the Tiger language, we just depend on the Tiger language implementation as we would depend on a regular Java library, create an instance of the TigerParser class, and then use that to parse a string into an AST. We still want to automatically provide integrations with the command-line, build systems such as Gradle, and IDEs such as Eclipse and IntelliJ. Therefore, every language implementation must implement the LanguageInstance interface. Spoofax 3 then provides libraries which take a LanguageInstance object, and integrate it with a platform. For example, spoofax.cli takes a LanguageInstance object and provides a command-line application, and spoofax.eclipse does the same for an Eclipse plugin. Because language implementations are just regular Java libraries, they now require some Java boilerplate. However, we do not want language developers to write this Java boilerplate for standard cases. Therefore, we employ a Spoofax 3 compiler that generates this Java boilerplate. If the language developer is not happy with the implementation, or wants to customize parts, they can manually implement or extend Java classes where needed. It is also possible to not use the Spoofax 3 compiler at all, and manually implement all parts. To enable quick language prototyping, we still support dynamic language loading in environments that support them (e.g., Eclipse and IntelliJ), by dynamically loading the language implementation JAR when changed. For example, when prototyping the Tiger language in Eclipse, if the syntax definition is changed we run the Spoofax 3 compiler to (incrementally) create a new parse table and Java classes, and dynamically (re)load the JAR. To improve the user experience, we use a configuration DSL to configure language specifications and implementations. Thereby configuration is centralized, has domain-specific checking, and editor services such as inline errors and code completion. We also allow changing of defaults (conventions), and persist them to enable renaming. To improve error traceability, errors are reported inline where possible. Errors are traced through PIE pipelines and support origin tracking to easily support error traceability and inline errors for all language implementations. TODO: better builders: non-Stratego commands incremental commands separate commands from how they are executed support command parameters/arguments continuous execution TODO: modular and incremental development of Spoofax 3 itself with Gradle","title":"Key ideas"},{"location":"background/motivation/","text":"Motivation \u00b6 In this section we discuss the motivations for developing Spoofax 3. Architecture \u00b6 The main motivation for developing Spoofax 3 is the monolithic, inflexible, and non-incremental architecture of Spoofax 2: It has an inflexible fixed-function pipeline , where every file of your language is parsed, analyzed, and transformed. This works fine, and can even be incremental when the files of your language can be separately compiled. However, this is often not the case. Languages should be able to define their own incremental pipelines with minimal effort. Those pipelines should be modular and flexible, enabling usage in a wide range applications such as command-line interfaces, build systems, code editors, and IDEs. It is monolithic for language users (i.e., the users of your programming language that you have developed with Spoofax), as every language developed with Spoofax 2 depends on Spoofax Core, which in turn depends on all meta-components: JSGLR1 and 2, NaBL+TS index and task engine, NaBL2 & Statix solver, dynsem interpreter, Stratego runtime, config parsing libraries, etc. A language should only require the meta-components that it uses. It is monolithic for meta-component developers (e.g., the developers of the language workbench, or researchers experimenting with new meta-tools or meta-languages). New meta-components need to be tightly integrated into Spoofax Core, requiring time-consuming changes and introducing increased coupling. We should develop meta-components in separation, and loosely couple/integrate them (as much as possible). The build of Spoofax 2 itself is monolithic and non-incremental, as all its components are compiled in one huge non-incremental build, massively increasing iteration time during development. The build must be incremental, and components should be separated where possible to reduce coupling, decreasing iteration times. Language loading \u00b6 Furthermore, Spoofax 2 only support dynamic loading of languages , where a language can be (re)loaded into the running environment. This is very useful during language development, as it enables fast prototyping. However, when we want to statically load the language, we still need to perform the dynamic loading ritual: somehow include the language archive in your application, and then load it at runtime. This hurts startup time, is not supported on some platforms (e.g., Graal native image), and is tedious. We should support both static and dynamic loading (where possible). Error tracing \u00b6 Some errors are not being traced back to their source. For example, many errors in configuration only show up during build time (in the console, which may be hidden) and are not traced back to the configuration file. This confuses users, and may get stuck on simple things, which then require help from us. Errors, warnings, and informational messages should be traced back to their source, and shown inline at the source in IDE environments, or shown as a list of messages with origin information on the command-line. When there are errors, execution should continue in certain instances (e.g., parse error should recover and try to do analysis), but should not in others (e.g., error from static analysis should prevent execution since it could crash). Configuration \u00b6 Another issue is the scattered configuration in language specifications, which is spread over many different places: metaborg.yaml editor/*.esv dynsem.properties In meta-languages files. For example, template options in SDF3. pom.xml .mvn/extensions.xml Finding the right configuration option in these files, and keeping them in sync, is tedious. Furthermore, while most configuration is documented on our website, looking that up still causes a cognitive gap. We should consolidate configuration that belongs together, and not have any duplicate configuration that needs to be kept in sync. Configuration should be supported with editor services such as inline errors and code completion, if possible. Moreover, some parts of a language specification are configured by convention, and these conventions cannot be changed. For example, the main SDF3 file is always assumed to be syntax/.sdf3 . When the language name is changed, but we forget to change the name of this main file, no parse table is built. Configuration conventions should be changeable, and defaults should be persisted to ensure that renamings do not break things. Summary of Problems \u00b6 To summarize, Spoofax 2 suffers from the following problems that form the motivation for Spoofax 3: Monolithic, inflexible, and non-incremental architecture causing: Inflexible and slow language processing due to non-incremental fixed-function pipeline Coupling in Spoofax Core: every language depends on Spoofax Core, and Spoofax Core depends on all meta-components Slow iteration times when developing Spoofax 2 due to its monolithic and non-incremental build Tedious to use languages due to dynamic language loading Confusing (end-)user experience due to: Bad error traceability Scattered configuration Non-incremental configuration (restarts required to update configuration)","title":"Motivation"},{"location":"background/motivation/#motivation","text":"In this section we discuss the motivations for developing Spoofax 3.","title":"Motivation"},{"location":"background/motivation/#architecture","text":"The main motivation for developing Spoofax 3 is the monolithic, inflexible, and non-incremental architecture of Spoofax 2: It has an inflexible fixed-function pipeline , where every file of your language is parsed, analyzed, and transformed. This works fine, and can even be incremental when the files of your language can be separately compiled. However, this is often not the case. Languages should be able to define their own incremental pipelines with minimal effort. Those pipelines should be modular and flexible, enabling usage in a wide range applications such as command-line interfaces, build systems, code editors, and IDEs. It is monolithic for language users (i.e., the users of your programming language that you have developed with Spoofax), as every language developed with Spoofax 2 depends on Spoofax Core, which in turn depends on all meta-components: JSGLR1 and 2, NaBL+TS index and task engine, NaBL2 & Statix solver, dynsem interpreter, Stratego runtime, config parsing libraries, etc. A language should only require the meta-components that it uses. It is monolithic for meta-component developers (e.g., the developers of the language workbench, or researchers experimenting with new meta-tools or meta-languages). New meta-components need to be tightly integrated into Spoofax Core, requiring time-consuming changes and introducing increased coupling. We should develop meta-components in separation, and loosely couple/integrate them (as much as possible). The build of Spoofax 2 itself is monolithic and non-incremental, as all its components are compiled in one huge non-incremental build, massively increasing iteration time during development. The build must be incremental, and components should be separated where possible to reduce coupling, decreasing iteration times.","title":"Architecture"},{"location":"background/motivation/#language-loading","text":"Furthermore, Spoofax 2 only support dynamic loading of languages , where a language can be (re)loaded into the running environment. This is very useful during language development, as it enables fast prototyping. However, when we want to statically load the language, we still need to perform the dynamic loading ritual: somehow include the language archive in your application, and then load it at runtime. This hurts startup time, is not supported on some platforms (e.g., Graal native image), and is tedious. We should support both static and dynamic loading (where possible).","title":"Language loading"},{"location":"background/motivation/#error-tracing","text":"Some errors are not being traced back to their source. For example, many errors in configuration only show up during build time (in the console, which may be hidden) and are not traced back to the configuration file. This confuses users, and may get stuck on simple things, which then require help from us. Errors, warnings, and informational messages should be traced back to their source, and shown inline at the source in IDE environments, or shown as a list of messages with origin information on the command-line. When there are errors, execution should continue in certain instances (e.g., parse error should recover and try to do analysis), but should not in others (e.g., error from static analysis should prevent execution since it could crash).","title":"Error tracing"},{"location":"background/motivation/#configuration","text":"Another issue is the scattered configuration in language specifications, which is spread over many different places: metaborg.yaml editor/*.esv dynsem.properties In meta-languages files. For example, template options in SDF3. pom.xml .mvn/extensions.xml Finding the right configuration option in these files, and keeping them in sync, is tedious. Furthermore, while most configuration is documented on our website, looking that up still causes a cognitive gap. We should consolidate configuration that belongs together, and not have any duplicate configuration that needs to be kept in sync. Configuration should be supported with editor services such as inline errors and code completion, if possible. Moreover, some parts of a language specification are configured by convention, and these conventions cannot be changed. For example, the main SDF3 file is always assumed to be syntax/.sdf3 . When the language name is changed, but we forget to change the name of this main file, no parse table is built. Configuration conventions should be changeable, and defaults should be persisted to ensure that renamings do not break things.","title":"Configuration"},{"location":"background/motivation/#summary-of-problems","text":"To summarize, Spoofax 2 suffers from the following problems that form the motivation for Spoofax 3: Monolithic, inflexible, and non-incremental architecture causing: Inflexible and slow language processing due to non-incremental fixed-function pipeline Coupling in Spoofax Core: every language depends on Spoofax Core, and Spoofax Core depends on all meta-components Slow iteration times when developing Spoofax 2 due to its monolithic and non-incremental build Tedious to use languages due to dynamic language loading Confusing (end-)user experience due to: Bad error traceability Scattered configuration Non-incremental configuration (restarts required to update configuration)","title":"Summary of Problems"},{"location":"background/status/","text":"Current Status \u00b6 We have stated our key ideas, but since Spoofax 3 is still under heavy development, they have not all been implemented yet. We now discuss the current status of Spoofax 3 by summarizing the key ideas and whether they has been implemented, along with any comments. Decoupling : Spoofax Core not depend on any meta-components. Language implementations instead depend on the meta-components they require. Flexible, modular and incremental pipelines : Use PIE . Static loading : Use static loading by default, making language implementation plain JAR files, which are easy to use in the Java ecosystem. LanguageInstance interface : Language implementations must implement the LanguageInstance interface, which a platform library uses to integrate a language with the platform. An initial version of the LanguageInstance interface exists, but this interface is not yet stable and will receive many new features. Currently, this interface contains features pertaining both command-line platforms and IDE/code editor platforms. These may be split up in the future. Generate Java boilerplate : Generate the Java boilerplate that Spoofax 3 now requires due to the LanguageInstance interface and language implementations being plain JAR files. Configuration for Spoofax 3 language implementations based on Spoofax 2 language definitions is provided through a Gradle build script, which is verbose. Quick language prototyping : Support dynamic language loading in environments that support this, to enable quick language prototyping. Configuration DSL : Use a configuration DSL to improve the developer/user experience. Error origin tracking : Perform origin tracking and propagation on errors to improve the developer/user experience. Not all PIE tasks trace errors, and some errors do not have location information yet. Commands : More flexible and incremental version of \"builders\" from Spoofax 2. Non-Stratego commands : Commands execute PIE tasks, which execute Java code. Incremental commands : Commands are incremental because they execute PIE tasks. Separate commands from how they are executed : Commands can be bound to IDE/editor menus, command-line commands, or to resource changes. Command parameters/arguments : Commands can specify parameters, which must be provided as arguments when executed. Modular and incremental development : Use Gradle (instead of Maven) to build Spoofax 3, which increases modularity and provides incremental builds for faster iteration times. Certain changes to core components may trigger long rebuilds, as a lot of projects (indirectly) depend on these core components and require recompilation. Certain changes trigger recompilation of Gradle plugins which are required by the rest of the build. This may cause a long configuration phase which is not parallelized. Our Gradle plugins do not support the Gradle build cache yet. Our Gradle plugins do not support the configuration cache yet. Sometimes multiple imports into IntelliJ are required to have it recognize all dependencies. Furthermore, we now discuss the status of features that were not new key ideas. Language builds Meta-language bootstrapping Bootstrapping requires implementation of the meta-languages in Spoofax 3, which we have not done yet. Meta-tools Syntax specification SDF3 Parsing JSGLR1 JSGLR2 Incremental parsing (but incompatible with recovery) Styling specification ESV (syntax-based) Semantic analysis NaBL2 Only supported for Spoofax 2-based language definitions Statix Statix signature generation based on SDF3 specification FlowSpec Stratego 2 Transformation (compilation) Stratego 2 Testing SPT Not all expectations have been ported over yet Editor services Syntax-based styling Inline error/warning/note messages Reference resolution Hover tooltips Code completion Syntactic Semantic (i.e., based on static semantics) Outline Platforms Command-line Eclipse Concurrency/parallelism is mostly ignored. Therefore, things may run concurrently that are not suppose to which cause data races and crashes. Several editor services and other conveniences are still missing or work in progress. IntelliJ A very minimal IntelliJ plugin for your language is provided, currently only supporting syntax highlighting and inline parse errors. Gradle Maven REPL The following features are being prototyped/experimented with Spoofax 3: Multi-lingual semantic analysis with Statix (Aron Zwaan) Semantic code completion based on Statix specification (Daniel Pelsmaeker) The following features will most likely not be supported: Analysis with NaBL/TS","title":"Current Status"},{"location":"background/status/#current-status","text":"We have stated our key ideas, but since Spoofax 3 is still under heavy development, they have not all been implemented yet. We now discuss the current status of Spoofax 3 by summarizing the key ideas and whether they has been implemented, along with any comments. Decoupling : Spoofax Core not depend on any meta-components. Language implementations instead depend on the meta-components they require. Flexible, modular and incremental pipelines : Use PIE . Static loading : Use static loading by default, making language implementation plain JAR files, which are easy to use in the Java ecosystem. LanguageInstance interface : Language implementations must implement the LanguageInstance interface, which a platform library uses to integrate a language with the platform. An initial version of the LanguageInstance interface exists, but this interface is not yet stable and will receive many new features. Currently, this interface contains features pertaining both command-line platforms and IDE/code editor platforms. These may be split up in the future. Generate Java boilerplate : Generate the Java boilerplate that Spoofax 3 now requires due to the LanguageInstance interface and language implementations being plain JAR files. Configuration for Spoofax 3 language implementations based on Spoofax 2 language definitions is provided through a Gradle build script, which is verbose. Quick language prototyping : Support dynamic language loading in environments that support this, to enable quick language prototyping. Configuration DSL : Use a configuration DSL to improve the developer/user experience. Error origin tracking : Perform origin tracking and propagation on errors to improve the developer/user experience. Not all PIE tasks trace errors, and some errors do not have location information yet. Commands : More flexible and incremental version of \"builders\" from Spoofax 2. Non-Stratego commands : Commands execute PIE tasks, which execute Java code. Incremental commands : Commands are incremental because they execute PIE tasks. Separate commands from how they are executed : Commands can be bound to IDE/editor menus, command-line commands, or to resource changes. Command parameters/arguments : Commands can specify parameters, which must be provided as arguments when executed. Modular and incremental development : Use Gradle (instead of Maven) to build Spoofax 3, which increases modularity and provides incremental builds for faster iteration times. Certain changes to core components may trigger long rebuilds, as a lot of projects (indirectly) depend on these core components and require recompilation. Certain changes trigger recompilation of Gradle plugins which are required by the rest of the build. This may cause a long configuration phase which is not parallelized. Our Gradle plugins do not support the Gradle build cache yet. Our Gradle plugins do not support the configuration cache yet. Sometimes multiple imports into IntelliJ are required to have it recognize all dependencies. Furthermore, we now discuss the status of features that were not new key ideas. Language builds Meta-language bootstrapping Bootstrapping requires implementation of the meta-languages in Spoofax 3, which we have not done yet. Meta-tools Syntax specification SDF3 Parsing JSGLR1 JSGLR2 Incremental parsing (but incompatible with recovery) Styling specification ESV (syntax-based) Semantic analysis NaBL2 Only supported for Spoofax 2-based language definitions Statix Statix signature generation based on SDF3 specification FlowSpec Stratego 2 Transformation (compilation) Stratego 2 Testing SPT Not all expectations have been ported over yet Editor services Syntax-based styling Inline error/warning/note messages Reference resolution Hover tooltips Code completion Syntactic Semantic (i.e., based on static semantics) Outline Platforms Command-line Eclipse Concurrency/parallelism is mostly ignored. Therefore, things may run concurrently that are not suppose to which cause data races and crashes. Several editor services and other conveniences are still missing or work in progress. IntelliJ A very minimal IntelliJ plugin for your language is provided, currently only supporting syntax highlighting and inline parse errors. Gradle Maven REPL The following features are being prototyped/experimented with Spoofax 3: Multi-lingual semantic analysis with Statix (Aron Zwaan) Semantic code completion based on Statix specification (Daniel Pelsmaeker) The following features will most likely not be supported: Analysis with NaBL/TS","title":"Current Status"},{"location":"guide/ask_for_help/","text":"Ask for help \u00b6 Before asking for help: try to troubleshoot the problem yourself . Performing troubleshooting makes you more familiar with the logging features of Spoofax, which can help you to improve the question you ask. check if the question has already been asked and/or answered in the Spoofax 3 Ask a question discussion forum by searching for your question. If not answered, continue discussion on that question. check if there is a bug report related to your question in the Spoofax 3 issue tracker by searching. Also check closed issues, as they may provide an answer to your question. If there is an open issue related to your question, continue discussion on that issue. Ask for help by creating a new discussion in the Spoofax 3 Ask a question discussion forum . Comprehensive and detailed questions are more likely to be answered. Try to include the following information in your question if applicable: Spoofax version Operating system & version Errors and stack traces (if applicable) Build logs (if applicable) Thread dumps (if applicable) Heap dumps (if applicable) Java version (if not using an embedded JVM) Eclipse version (if not using the standard Eclipse installation) Gradle version (if applicable) Also, please consider helping other members by answering questions if you know the answer!","title":"Ask for help"},{"location":"guide/ask_for_help/#ask-for-help","text":"Before asking for help: try to troubleshoot the problem yourself . Performing troubleshooting makes you more familiar with the logging features of Spoofax, which can help you to improve the question you ask. check if the question has already been asked and/or answered in the Spoofax 3 Ask a question discussion forum by searching for your question. If not answered, continue discussion on that question. check if there is a bug report related to your question in the Spoofax 3 issue tracker by searching. Also check closed issues, as they may provide an answer to your question. If there is an open issue related to your question, continue discussion on that issue. Ask for help by creating a new discussion in the Spoofax 3 Ask a question discussion forum . Comprehensive and detailed questions are more likely to be answered. Try to include the following information in your question if applicable: Spoofax version Operating system & version Errors and stack traces (if applicable) Build logs (if applicable) Thread dumps (if applicable) Heap dumps (if applicable) Java version (if not using an embedded JVM) Eclipse version (if not using the standard Eclipse installation) Gradle version (if applicable) Also, please consider helping other members by answering questions if you know the answer!","title":"Ask for help"},{"location":"guide/report_a_bug/","text":"Report a bug \u00b6 Before reporting a bug: check if there is an existing bug report in the Spoofax 3 issue tracker by searching. Also check closed issues, as the bug may already be fixed or closed for other reasons. If there is already an open issue for the bug, continue discussion on that issue. Report the bug by creating a new issue in the Spoofax 3 issue tracker . Make sure to fill in the template, and to be comprehensive and detailed, to make it easier for us to resolve the bug report. The troubleshooting guide can help you with getting exception stack traces, logs, thread dumps, and heap dumps. Please include these with the bug report if you think they are applicable.","title":"Report a bug"},{"location":"guide/report_a_bug/#report-a-bug","text":"Before reporting a bug: check if there is an existing bug report in the Spoofax 3 issue tracker by searching. Also check closed issues, as the bug may already be fixed or closed for other reasons. If there is already an open issue for the bug, continue discussion on that issue. Report the bug by creating a new issue in the Spoofax 3 issue tracker . Make sure to fill in the template, and to be comprehensive and detailed, to make it easier for us to resolve the bug report. The troubleshooting guide can help you with getting exception stack traces, logs, thread dumps, and heap dumps. Please include these with the bug report if you think they are applicable.","title":"Report a bug"},{"location":"guide/development/debugging-in-intellij/","text":"How to debug Spoofax 3 development using IntelliJ \u00b6 To debug Spoofax 3 development using IntelliJ, create a new Run Configuration and pass the --debug-jvm option to the task. For example: :spoofax3.example.root:tiger.eclipse:runEclipse --debug-jvm Error: Unknown command-line option '--debug-jvm' You specified the delegate task name (e.g., runTigerEclipse ) instead of the full task name. Only the JavaExec type tasks support the --debug-jvm option, and the delegate tasks are of the wrong type. Specify the full task name instead. For example, if the delegate task runTigerEclipse is defined in the root build.gradle.kts like this: build.gradle.kts tasksWithIncludedBuild ( \"spoofax3.example.root\" ) { registerDelegateTask ( \"runTigerCli\" , it , \":tiger.cli:run\" ) registerDelegateTask ( \"runTigerEclipse\" , it , \":tiger.eclipse:runEclipse\" ) registerDelegateTask ( \"runTigerIntelliJ\" , it , \":tiger.intellij:runIde\" ) } Then the full task name is the name of the included build combined with the name of the task, starting and separated with colons ( : ). Thus it would be: :spoofax3.example.root:tiger.eclipse:runEclipse Then start the configuration in Run mode, and wait until the following shows up in the console: Listening for transport dt_socket at address: 5005 [Attach debugger] Then click the Attach debugger text in the console to start and attach an external debugger and start debugging. Debugging tests \u00b6 If you are debugging tests, make sure that the test results are cleaned before by running cleanTest , otherwise Gradle may skip the test task. For example, run the following Gradle tasks as part of the Run configuration: :spoofax3.lwb.root:spoofax.dynamicloading:cleanTest :spoofax3.lwb.root:spoofax.dynamicloading:test What about JAVA_TOOL_OPTIONS ? \u00b6 Instead of specifying --debug-jvm , you can add the following configuration variable to your Run configuration: JAVA_TOOL_OPTIONS=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 This will work regardless of which task is specified, including specifying a delegate (non JavaExec ) task. However, note an important downside with this approach: this enables debugging for any Gradle task that executes Java in an isolated way, including any (Java/Kotlin) compilation tasks that run in a separate process. You might see the following error: Picked up JAVA_TOOL_OPTIONS: -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 ERROR: transport error 202: bind failed: Address already in use FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197) ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [debugInit.c:750] To avoid this error, make sure to first build normally such that these tasks are no longer executed, then run your debugging configuration.","title":"Debugging in IntelliJ"},{"location":"guide/development/debugging-in-intellij/#how-to-debug-spoofax-3-development-using-intellij","text":"To debug Spoofax 3 development using IntelliJ, create a new Run Configuration and pass the --debug-jvm option to the task. For example: :spoofax3.example.root:tiger.eclipse:runEclipse --debug-jvm Error: Unknown command-line option '--debug-jvm' You specified the delegate task name (e.g., runTigerEclipse ) instead of the full task name. Only the JavaExec type tasks support the --debug-jvm option, and the delegate tasks are of the wrong type. Specify the full task name instead. For example, if the delegate task runTigerEclipse is defined in the root build.gradle.kts like this: build.gradle.kts tasksWithIncludedBuild ( \"spoofax3.example.root\" ) { registerDelegateTask ( \"runTigerCli\" , it , \":tiger.cli:run\" ) registerDelegateTask ( \"runTigerEclipse\" , it , \":tiger.eclipse:runEclipse\" ) registerDelegateTask ( \"runTigerIntelliJ\" , it , \":tiger.intellij:runIde\" ) } Then the full task name is the name of the included build combined with the name of the task, starting and separated with colons ( : ). Thus it would be: :spoofax3.example.root:tiger.eclipse:runEclipse Then start the configuration in Run mode, and wait until the following shows up in the console: Listening for transport dt_socket at address: 5005 [Attach debugger] Then click the Attach debugger text in the console to start and attach an external debugger and start debugging.","title":"How to debug Spoofax 3 development using IntelliJ"},{"location":"guide/development/debugging-in-intellij/#debugging-tests","text":"If you are debugging tests, make sure that the test results are cleaned before by running cleanTest , otherwise Gradle may skip the test task. For example, run the following Gradle tasks as part of the Run configuration: :spoofax3.lwb.root:spoofax.dynamicloading:cleanTest :spoofax3.lwb.root:spoofax.dynamicloading:test","title":"Debugging tests"},{"location":"guide/development/debugging-in-intellij/#what-about-java_tool_options","text":"Instead of specifying --debug-jvm , you can add the following configuration variable to your Run configuration: JAVA_TOOL_OPTIONS=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 This will work regardless of which task is specified, including specifying a delegate (non JavaExec ) task. However, note an important downside with this approach: this enables debugging for any Gradle task that executes Java in an isolated way, including any (Java/Kotlin) compilation tasks that run in a separate process. You might see the following error: Picked up JAVA_TOOL_OPTIONS: -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 ERROR: transport error 202: bind failed: Address already in use FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197) ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [debugInit.c:750] To avoid this error, make sure to first build normally such that these tasks are no longer executed, then run your debugging configuration.","title":"What about JAVA_TOOL_OPTIONS?"},{"location":"guide/development/troubleshooting/","text":"Troubleshooting \u00b6 This guide explains how to troubleshoot issues when developing Spoofax itself. In general, ensure you're calling ./repo and ./gradlew on Linux and MacOS (or repo.bat and gradlew.bat on Windows) instead of your local Gradle installation. The local one may be too old or too new. Known Problems \u00b6 The following are known problems that can occur, and their solutions or workarounds. Many errors about unresolved classes in IntelliJ \u00b6 If after importing there are many errors in files about classes not existing, re-import all the projects by pressing the Reload All Gradle Projects button in the Gradle tool window. Cannot debug in IntelliJ \u00b6 See how to debug in IntelliJ for more information and tips. Profiling in IntelliJ \u00b6 Profiling in IntelliJ can be done similarly to debugging. For example, to profile with YourKit, add the following environment variable to your run configuration: JAVA_TOOL_OPTIONS=-agentpath:/Applications/YourKit-Java-Profiler-2020.9.app/Contents/Resources/bin/mac/libyjpagent.dylib=listen=all,sampling,onexit=snapshot If you are using a different profiler, the agentpath needs to point to the corresponding agent of your profiler, and the settings after the agent will need to be tailored towards your profiler. In the example above, the YourKit profiler will attach to the program, enable CPU sampling, and create a snapshot when the program ends. The snapshot can then be opened and inspected in YourKit. Similar to debugging, this enables profiling for any Gradle task that executes Java in an isolated way, and tests must be cleaned before profiling to force tests to be executed. Spoofax 2 language fails to build with \"Previous build failed and no change in the build input has been observed\" \u00b6 If building a Spoofax 2 language fails due to some ephemeral issue, or if building is cancelled (because you cancelled the Gradle build), the following exception may be thrown during the build: org.metaborg.core.MetaborgException: Previous build failed and no change in the build input has been observed, not rebuilding. Fix the problem, or clean and rebuild the project to force a rebuild This is an artefact of the Pluto build system refusing to rebuild if it failed but no changes to the input were detected. To force Pluto to rebuild, delete the target/pluto directory of the language. Task 'buildAll' not found in root project 'devenv' \u00b6 You have 'configure on demand' enabled, such as org.gradle.configureondemand=true in your ~/.gradle/gradle.properties file. Disable this. Expiring Daemon because JVM heap space is exhausted \u00b6 The memory limits in gradle.properties may be too low, and may need to be increased. Running the build without --parallel may decrease memory pressure, as less tasks are executed concurrently. Or, there is a memory leak in the build: please make a heap dump and send this to the developers so it can be addressed. Could not create service of type FileAccessTimeJournal using GradleUserHomeScopeServices.createFileAccessTimeJournal() \u00b6 The permissions in your ~/.gradle/ directory are too restrictive. For example, if you're using WSL, ensure the directory is not a symlink to the Windows' .gradle/ directory. Error resolving plugin: Plugin request for plugin already on the classpath must not include a version \u00b6 Error resolving plugin [id: 'org.metaborg.gradle.config.devenv', version: '?'] Plugin request for plugin already on the classpath must not include a version You are not running with the recommended version of Gradle. Unknown command-line option '--args' \u00b6 Command-line arguments such as --args are not supported for tasks in the root project, such as the runSdf3Cli task. Instead, go to the relevant included build and call the task directly. cd spoofax.pie/example ./gradlew :sdf3.cli:run --args=\"-V\" The working directory is the directory with the gradle.build.kts file of the CLI project. This cannot be changed. For example, spoofax.pie/example/sdf3/sdf3.cli/ for the :sdf3.cli project. Failed to create Jar file \u00b6 Due to a bug in the Java 8 and 9 compiler, it sometimes generates the wrong byte code for annotations . This, in turn, causes Gradle to fail reading the JAR file with the following error: A problem occurred configuring project ':spoofax3.example.root'. > Failed to create Jar file /Users/username/.gradle/caches/jars-8/defabc/jsglr.common-develop-SNAPSHOT.jar. The stack trace will look something like this: org.gradle.api.ProjectConfigurationException: A problem occurred configuring project ':spoofax3.example.root'. at org.gradle.configuration.project.LifecycleProjectEvaluator.wrapException(LifecycleProjectEvaluator.java:75) at org.gradle.configuration.project.LifecycleProjectEvaluator.addConfigurationFailure(LifecycleProjectEvaluator.java:68) at org.gradle.configuration.project.LifecycleProjectEvaluator.access$400(LifecycleProjectEvaluator.java:51) ... 142 more Caused by: org.gradle.api.GradleException: Failed to create Jar file /Users/username/.gradle/caches/jars-8/defabc/jsglr.common-develop-SNAPSHOT.jar. at org.gradle.internal.classpath.ClasspathBuilder.jar(ClasspathBuilder.java:47) at org.gradle.internal.classpath.InstrumentingClasspathFileTransformer.instrument(InstrumentingClasspathFileTransformer.java:83) ... 6 more Caused by: java.lang.ArrayIndexOutOfBoundsException: 195 at org.objectweb.asm.ClassReader.readLabel(ClassReader.java:2654) at org.objectweb.asm.ClassReader.createLabel(ClassReader.java:2670) at org.objectweb.asm.ClassReader.readTypeAnnotations(ClassReader.java:2736) at org.objectweb.asm.ClassReader.readCode(ClassReader.java:1912) at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1492) at org.objectweb.asm.ClassReader.accept(ClassReader.java:717) ... 15 more Determine the version of Java using java -version . Java 8 and 9 can exhibit this problem. The solution is to update your Java to version 11 or later. You can use a tool such as SDKMAN! to easily manage the (default) versions of Java on your system.","title":"Troubleshooting"},{"location":"guide/development/troubleshooting/#troubleshooting","text":"This guide explains how to troubleshoot issues when developing Spoofax itself. In general, ensure you're calling ./repo and ./gradlew on Linux and MacOS (or repo.bat and gradlew.bat on Windows) instead of your local Gradle installation. The local one may be too old or too new.","title":"Troubleshooting"},{"location":"guide/development/troubleshooting/#known-problems","text":"The following are known problems that can occur, and their solutions or workarounds.","title":"Known Problems"},{"location":"guide/development/troubleshooting/#many-errors-about-unresolved-classes-in-intellij","text":"If after importing there are many errors in files about classes not existing, re-import all the projects by pressing the Reload All Gradle Projects button in the Gradle tool window.","title":"Many errors about unresolved classes in IntelliJ"},{"location":"guide/development/troubleshooting/#cannot-debug-in-intellij","text":"See how to debug in IntelliJ for more information and tips.","title":"Cannot debug in IntelliJ"},{"location":"guide/development/troubleshooting/#profiling-in-intellij","text":"Profiling in IntelliJ can be done similarly to debugging. For example, to profile with YourKit, add the following environment variable to your run configuration: JAVA_TOOL_OPTIONS=-agentpath:/Applications/YourKit-Java-Profiler-2020.9.app/Contents/Resources/bin/mac/libyjpagent.dylib=listen=all,sampling,onexit=snapshot If you are using a different profiler, the agentpath needs to point to the corresponding agent of your profiler, and the settings after the agent will need to be tailored towards your profiler. In the example above, the YourKit profiler will attach to the program, enable CPU sampling, and create a snapshot when the program ends. The snapshot can then be opened and inspected in YourKit. Similar to debugging, this enables profiling for any Gradle task that executes Java in an isolated way, and tests must be cleaned before profiling to force tests to be executed.","title":"Profiling in IntelliJ"},{"location":"guide/development/troubleshooting/#spoofax-2-language-fails-to-build-with-previous-build-failed-and-no-change-in-the-build-input-has-been-observed","text":"If building a Spoofax 2 language fails due to some ephemeral issue, or if building is cancelled (because you cancelled the Gradle build), the following exception may be thrown during the build: org.metaborg.core.MetaborgException: Previous build failed and no change in the build input has been observed, not rebuilding. Fix the problem, or clean and rebuild the project to force a rebuild This is an artefact of the Pluto build system refusing to rebuild if it failed but no changes to the input were detected. To force Pluto to rebuild, delete the target/pluto directory of the language.","title":"Spoofax 2 language fails to build with \"Previous build failed and no change in the build input has been observed\""},{"location":"guide/development/troubleshooting/#task-buildall-not-found-in-root-project-devenv","text":"You have 'configure on demand' enabled, such as org.gradle.configureondemand=true in your ~/.gradle/gradle.properties file. Disable this.","title":"Task 'buildAll' not found in root project 'devenv'"},{"location":"guide/development/troubleshooting/#expiring-daemon-because-jvm-heap-space-is-exhausted","text":"The memory limits in gradle.properties may be too low, and may need to be increased. Running the build without --parallel may decrease memory pressure, as less tasks are executed concurrently. Or, there is a memory leak in the build: please make a heap dump and send this to the developers so it can be addressed.","title":"Expiring Daemon because JVM heap space is exhausted"},{"location":"guide/development/troubleshooting/#could-not-create-service-of-type-fileaccesstimejournal-using-gradleuserhomescopeservicescreatefileaccesstimejournal","text":"The permissions in your ~/.gradle/ directory are too restrictive. For example, if you're using WSL, ensure the directory is not a symlink to the Windows' .gradle/ directory.","title":"Could not create service of type FileAccessTimeJournal using GradleUserHomeScopeServices.createFileAccessTimeJournal()"},{"location":"guide/development/troubleshooting/#error-resolving-plugin-plugin-request-for-plugin-already-on-the-classpath-must-not-include-a-version","text":"Error resolving plugin [id: 'org.metaborg.gradle.config.devenv', version: '?'] Plugin request for plugin already on the classpath must not include a version You are not running with the recommended version of Gradle.","title":"Error resolving plugin: Plugin request for plugin already on the classpath must not include a version"},{"location":"guide/development/troubleshooting/#unknown-command-line-option-args","text":"Command-line arguments such as --args are not supported for tasks in the root project, such as the runSdf3Cli task. Instead, go to the relevant included build and call the task directly. cd spoofax.pie/example ./gradlew :sdf3.cli:run --args=\"-V\" The working directory is the directory with the gradle.build.kts file of the CLI project. This cannot be changed. For example, spoofax.pie/example/sdf3/sdf3.cli/ for the :sdf3.cli project.","title":"Unknown command-line option '--args'"},{"location":"guide/development/troubleshooting/#failed-to-create-jar-file","text":"Due to a bug in the Java 8 and 9 compiler, it sometimes generates the wrong byte code for annotations . This, in turn, causes Gradle to fail reading the JAR file with the following error: A problem occurred configuring project ':spoofax3.example.root'. > Failed to create Jar file /Users/username/.gradle/caches/jars-8/defabc/jsglr.common-develop-SNAPSHOT.jar. The stack trace will look something like this: org.gradle.api.ProjectConfigurationException: A problem occurred configuring project ':spoofax3.example.root'. at org.gradle.configuration.project.LifecycleProjectEvaluator.wrapException(LifecycleProjectEvaluator.java:75) at org.gradle.configuration.project.LifecycleProjectEvaluator.addConfigurationFailure(LifecycleProjectEvaluator.java:68) at org.gradle.configuration.project.LifecycleProjectEvaluator.access$400(LifecycleProjectEvaluator.java:51) ... 142 more Caused by: org.gradle.api.GradleException: Failed to create Jar file /Users/username/.gradle/caches/jars-8/defabc/jsglr.common-develop-SNAPSHOT.jar. at org.gradle.internal.classpath.ClasspathBuilder.jar(ClasspathBuilder.java:47) at org.gradle.internal.classpath.InstrumentingClasspathFileTransformer.instrument(InstrumentingClasspathFileTransformer.java:83) ... 6 more Caused by: java.lang.ArrayIndexOutOfBoundsException: 195 at org.objectweb.asm.ClassReader.readLabel(ClassReader.java:2654) at org.objectweb.asm.ClassReader.createLabel(ClassReader.java:2670) at org.objectweb.asm.ClassReader.readTypeAnnotations(ClassReader.java:2736) at org.objectweb.asm.ClassReader.readCode(ClassReader.java:1912) at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1492) at org.objectweb.asm.ClassReader.accept(ClassReader.java:717) ... 15 more Determine the version of Java using java -version . Java 8 and 9 can exhibit this problem. The solution is to update your Java to version 11 or later. You can use a tool such as SDKMAN! to easily manage the (default) versions of Java on your system.","title":"Failed to create Jar file"},{"location":"guide/eclipse_lwb/import/","text":"Importing a Project \u00b6 An existing Spoofax 3 language project can be imported into Eclipse as follows. In the main menu of Eclipse, select File \u2023 Import... to open the import dialog. In the import dialog, select General \u2023 Existing Projects into Workspace and press Next > to open the import projects dialog. In the import projects dialog, ensure Select root directory: is ticked and press Browse... on the right of that. Select the root directory of the language project that you want to import (the directory that contains spoofaxc.cfg ) and press Open . Then, press Finish to import the project. The project should now be imported into your workspace. Finally, build the language project by selecting the project in the Package Explorer and choosing Project \u2023 Build Project . Building the project at least once after importing is required to update the project. Troubleshooting \u00b6 The project does not appear in the Import window \u00b6 Ensure the project has associated .project and .classpath files, and that they are not ignored for version control. Minimum file content .","title":"Importing a Project"},{"location":"guide/eclipse_lwb/import/#importing-a-project","text":"An existing Spoofax 3 language project can be imported into Eclipse as follows. In the main menu of Eclipse, select File \u2023 Import... to open the import dialog. In the import dialog, select General \u2023 Existing Projects into Workspace and press Next > to open the import projects dialog. In the import projects dialog, ensure Select root directory: is ticked and press Browse... on the right of that. Select the root directory of the language project that you want to import (the directory that contains spoofaxc.cfg ) and press Open . Then, press Finish to import the project. The project should now be imported into your workspace. Finally, build the language project by selecting the project in the Package Explorer and choosing Project \u2023 Build Project . Building the project at least once after importing is required to update the project.","title":"Importing a Project"},{"location":"guide/eclipse_lwb/import/#troubleshooting","text":"","title":"Troubleshooting"},{"location":"guide/eclipse_lwb/import/#the-project-does-not-appear-in-the-import-window","text":"Ensure the project has associated .project and .classpath files, and that they are not ignored for version control. Minimum file content .","title":"The project does not appear in the Import window"},{"location":"guide/eclipse_lwb/troubleshooting/","text":"Troubleshooting \u00b6 This how-to guide explains how to troubleshoot issues when building and testing languages in the Spoofax Eclipse LWB plugin. Diagnosing problems \u00b6 Looking for errors \u00b6 If something is not working as expected, the first thing to check is whether there are errors in the project by looking in the Project Explorer or Package Explorer view, which is open on the left-hand side of the IDE by default. If there are any red markers on the project, or any directory or file in the project, that indicates an error. When there are errors on files that define your language (e.g., .cfg/.sdf3/.stx/.str2 files), the language is not compiled and reloaded, so building the language has no effect, and your change will not be taken into account. Errors on example and test (e.g., .spt) files are ok, and do not prevent the language from being compiled and reloaded. It is also possible to get a list of all the errors in the project by opening the Problems view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Problems from the main menu. Consulting the logs \u00b6 Spoofax logs a lot of information to make troubleshooting easier. First consult the Error Log view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Error Log from the main menu. The error log contains warning and error events from all plugins running in Eclipse, including several Spoofax plugins. Most errors include stack traces, which help the Spoofax developers immensely in bug reports or when asking for help. Browse through the errors from Spoofax plugins to see if it can help you troubleshoot the problem. If nothing relevant is in the error log, try to consult the Console view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Other... \u2023 search for Console \u2023 choose Console and open it from the main menu. If the console view does not say Spoofax in the top-left corner, ensure the Spoofax console is open by pressing the small downward error on the blue monitor icon and selecting Spoofax console (see screenshot). Browse through the log from Spoofax plugins to see if it can help you troubleshoot the problem. When building, Spoofax logs all tasks and files that are checked, and all tasks that are built. At the end of the build, Spoofax logs whether it completed or if something went wrong. Rebuild your language and check the build log to see if it can help you troubleshoot the problem. If you ask for help or report a bug, consider storing the log in a text file, so you can include it in your help request or bug report. Problems and solutions \u00b6 Errors in language definition files \u00b6 Error markers in language definition files (e.g., .cfg/.sdf3/.stx/.str2 files) are expected. If there are errors on language definition files, solve them build the language. Errors in generated files \u00b6 If there are errors in generated files (i.e., files in the build/generated directory), first try to build the language, as building the language may re-generate these files. If the error persists, try deleting the generated file and then building the language. Also consider reporting this bug , as we consider incrementality issues like these bugs. If after deleting the generated file and rebuilding the language, the generated file comes back with errors, definitely report this as a bug . Try to work around the problem first by cleaning the language project or by deleting on-disk cache and restarting Eclipse Errors occurred during the build \u00b6 If after building a language, an error popup appears, something unexpected went wrong. Please report this as a bug . Try to work around the problem first by cleaning the language project or by deleting on-disk cache and restarting Eclipse Many duplicate definition errors/other weird errors \u00b6 If there are many errors about duplicate definitions, things already being defined, or other weird errors, see if there is a bin directory in your language project. If so, delete the bin directory and rebuild the project. Workarounds \u00b6 Cleaning the language project \u00b6 Try cleaning the language project by first selecting (clicking) the language project in the Project Explorer or Package Explorer view, and then choosing Project \u2023 Clean... from the main menu. In the clean window, deselect Clean all projects , select your project, and press Clean . Then, rebuild your language. Optionally, check the error log and console again. If this does not help, you can try to first delete the build directory, then clean the language project, and then build the project. If this solves your problem, consider reporting this bug , as we consider incrementality issues like these bugs. If not, try the next workaround. Deleting on-disk cache and restarting Eclipse \u00b6 Finally, there may be a problem related to the on-disk cache. First, close Eclipse. Then, navigate to your workspace directory and delete the .metadata/.plugins/spoofax.lwb.eclipse/pieStore file. These directories are hidden, so you may need to enable showing hidden files, or delete the file using your terminal. Then, start Eclipse again and build your language. Optionally, check the error log and console again. If this solves your problem, consider reporting this bug , as we consider incrementality issues like these bugs. If not, ask for help . Report a bug or ask for help \u00b6 If after troubleshooting the issue is not resolved, report this bug if you think this is a bug, or ask for help . Advanced troubleshooting \u00b6 Checking for deadlocks (and making a thread dump) \u00b6 If Eclipse seems to be stuck, hanging, or not making any progress, check for deadlocks by making a thread dump. First, we need to figure out the process ID of the JVM that is running eclipse. Run the jps command in a terminal. It will print something like: 80035 Jps 94805 74294 org.eclipse.equinox.launcher_1.6.100.v20201223-0822.jar 51515 Eclipse 74236 GradleDaemon In this case, 51515 is the process ID of Eclipse. Then run jstack , so jstack 51515 in this case. jstack prints a thread dump with a stack trace for each thread, alongside any detected deadlocks. While it may be hard to use this information to troubleshoot yourself, this can be useful information when asking for help or when reporting a bug. In case of deadlocks, please report this bug . Checking memory (and making a heap dump) \u00b6 If Eclipse seems to be using excessive amounts of memory or processor time, check how much heap space Eclipse is using. To show the heap space Eclipse is using, go to the Eclipse preferences, and in the General tab, enable Show heap status . The heap status shows up in the bottom right corner. Press the trash can icon to run garbage collection, which will free up any available memory. If after garbage collection, the memory is still near its maximum, Eclipse has run out of memory and will become very slow or unresponsive. To diagnose the problem, first make a thread dump as was described in the previous section. A thread dump may give some clues as to what is generating or leaking heap memory. Then, make a heap dump by running jmap -dump:live,format=b,file=heap.bin using the process ID of Eclipse, as was described in the previous section. This creates a heap dump file called heap.bin in the working directory. This heap dump can then be loaded into a profiler such as VisualVM for inspection. In case of excessive memory problems, please report this bug and share the heap dump. To share heap dumps, upload them to a cloud service such as Mega and share the link. Warning Heap dumps contain all the JVM memory in Eclipse, which can include any information that you have entered into Eclipse. Only share a heap dump if you are sure you have not entered any sensitive information into Eclipse.","title":"Troubleshooting"},{"location":"guide/eclipse_lwb/troubleshooting/#troubleshooting","text":"This how-to guide explains how to troubleshoot issues when building and testing languages in the Spoofax Eclipse LWB plugin.","title":"Troubleshooting"},{"location":"guide/eclipse_lwb/troubleshooting/#diagnosing-problems","text":"","title":"Diagnosing problems"},{"location":"guide/eclipse_lwb/troubleshooting/#looking-for-errors","text":"If something is not working as expected, the first thing to check is whether there are errors in the project by looking in the Project Explorer or Package Explorer view, which is open on the left-hand side of the IDE by default. If there are any red markers on the project, or any directory or file in the project, that indicates an error. When there are errors on files that define your language (e.g., .cfg/.sdf3/.stx/.str2 files), the language is not compiled and reloaded, so building the language has no effect, and your change will not be taken into account. Errors on example and test (e.g., .spt) files are ok, and do not prevent the language from being compiled and reloaded. It is also possible to get a list of all the errors in the project by opening the Problems view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Problems from the main menu.","title":"Looking for errors"},{"location":"guide/eclipse_lwb/troubleshooting/#consulting-the-logs","text":"Spoofax logs a lot of information to make troubleshooting easier. First consult the Error Log view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Error Log from the main menu. The error log contains warning and error events from all plugins running in Eclipse, including several Spoofax plugins. Most errors include stack traces, which help the Spoofax developers immensely in bug reports or when asking for help. Browse through the errors from Spoofax plugins to see if it can help you troubleshoot the problem. If nothing relevant is in the error log, try to consult the Console view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Other... \u2023 search for Console \u2023 choose Console and open it from the main menu. If the console view does not say Spoofax in the top-left corner, ensure the Spoofax console is open by pressing the small downward error on the blue monitor icon and selecting Spoofax console (see screenshot). Browse through the log from Spoofax plugins to see if it can help you troubleshoot the problem. When building, Spoofax logs all tasks and files that are checked, and all tasks that are built. At the end of the build, Spoofax logs whether it completed or if something went wrong. Rebuild your language and check the build log to see if it can help you troubleshoot the problem. If you ask for help or report a bug, consider storing the log in a text file, so you can include it in your help request or bug report.","title":"Consulting the logs"},{"location":"guide/eclipse_lwb/troubleshooting/#problems-and-solutions","text":"","title":"Problems and solutions"},{"location":"guide/eclipse_lwb/troubleshooting/#errors-in-language-definition-files","text":"Error markers in language definition files (e.g., .cfg/.sdf3/.stx/.str2 files) are expected. If there are errors on language definition files, solve them build the language.","title":"Errors in language definition files"},{"location":"guide/eclipse_lwb/troubleshooting/#errors-in-generated-files","text":"If there are errors in generated files (i.e., files in the build/generated directory), first try to build the language, as building the language may re-generate these files. If the error persists, try deleting the generated file and then building the language. Also consider reporting this bug , as we consider incrementality issues like these bugs. If after deleting the generated file and rebuilding the language, the generated file comes back with errors, definitely report this as a bug . Try to work around the problem first by cleaning the language project or by deleting on-disk cache and restarting Eclipse","title":"Errors in generated files"},{"location":"guide/eclipse_lwb/troubleshooting/#errors-occurred-during-the-build","text":"If after building a language, an error popup appears, something unexpected went wrong. Please report this as a bug . Try to work around the problem first by cleaning the language project or by deleting on-disk cache and restarting Eclipse","title":"Errors occurred during the build"},{"location":"guide/eclipse_lwb/troubleshooting/#many-duplicate-definition-errorsother-weird-errors","text":"If there are many errors about duplicate definitions, things already being defined, or other weird errors, see if there is a bin directory in your language project. If so, delete the bin directory and rebuild the project.","title":"Many duplicate definition errors/other weird errors"},{"location":"guide/eclipse_lwb/troubleshooting/#workarounds","text":"","title":"Workarounds"},{"location":"guide/eclipse_lwb/troubleshooting/#clean_project","text":"Try cleaning the language project by first selecting (clicking) the language project in the Project Explorer or Package Explorer view, and then choosing Project \u2023 Clean... from the main menu. In the clean window, deselect Clean all projects , select your project, and press Clean . Then, rebuild your language. Optionally, check the error log and console again. If this does not help, you can try to first delete the build directory, then clean the language project, and then build the project. If this solves your problem, consider reporting this bug , as we consider incrementality issues like these bugs. If not, try the next workaround.","title":"Cleaning the language project"},{"location":"guide/eclipse_lwb/troubleshooting/#clean_cache","text":"Finally, there may be a problem related to the on-disk cache. First, close Eclipse. Then, navigate to your workspace directory and delete the .metadata/.plugins/spoofax.lwb.eclipse/pieStore file. These directories are hidden, so you may need to enable showing hidden files, or delete the file using your terminal. Then, start Eclipse again and build your language. Optionally, check the error log and console again. If this solves your problem, consider reporting this bug , as we consider incrementality issues like these bugs. If not, ask for help .","title":"Deleting on-disk cache and restarting Eclipse"},{"location":"guide/eclipse_lwb/troubleshooting/#report-a-bug-or-ask-for-help","text":"If after troubleshooting the issue is not resolved, report this bug if you think this is a bug, or ask for help .","title":"Report a bug or ask for help"},{"location":"guide/eclipse_lwb/troubleshooting/#advanced-troubleshooting","text":"","title":"Advanced troubleshooting"},{"location":"guide/eclipse_lwb/troubleshooting/#checking-for-deadlocks-and-making-a-thread-dump","text":"If Eclipse seems to be stuck, hanging, or not making any progress, check for deadlocks by making a thread dump. First, we need to figure out the process ID of the JVM that is running eclipse. Run the jps command in a terminal. It will print something like: 80035 Jps 94805 74294 org.eclipse.equinox.launcher_1.6.100.v20201223-0822.jar 51515 Eclipse 74236 GradleDaemon In this case, 51515 is the process ID of Eclipse. Then run jstack , so jstack 51515 in this case. jstack prints a thread dump with a stack trace for each thread, alongside any detected deadlocks. While it may be hard to use this information to troubleshoot yourself, this can be useful information when asking for help or when reporting a bug. In case of deadlocks, please report this bug .","title":"Checking for deadlocks (and making a thread dump)"},{"location":"guide/eclipse_lwb/troubleshooting/#checking-memory-and-making-a-heap-dump","text":"If Eclipse seems to be using excessive amounts of memory or processor time, check how much heap space Eclipse is using. To show the heap space Eclipse is using, go to the Eclipse preferences, and in the General tab, enable Show heap status . The heap status shows up in the bottom right corner. Press the trash can icon to run garbage collection, which will free up any available memory. If after garbage collection, the memory is still near its maximum, Eclipse has run out of memory and will become very slow or unresponsive. To diagnose the problem, first make a thread dump as was described in the previous section. A thread dump may give some clues as to what is generating or leaking heap memory. Then, make a heap dump by running jmap -dump:live,format=b,file=heap.bin using the process ID of Eclipse, as was described in the previous section. This creates a heap dump file called heap.bin in the working directory. This heap dump can then be loaded into a profiler such as VisualVM for inspection. In case of excessive memory problems, please report this bug and share the heap dump. To share heap dumps, upload them to a cloud service such as Mega and share the link. Warning Heap dumps contain all the JVM memory in Eclipse, which can include any information that you have entered into Eclipse. Only share a heap dump if you are sure you have not entered any sensitive information into Eclipse.","title":"Checking memory (and making a heap dump)"},{"location":"guide/eclipse_lwb/update/","text":"Updating & Downgrading \u00b6 Updating \u00b6 The Spoofax 3 language workbench Eclipse plugin can be updated as follows. In the main menu of Eclipse, select Help \u2023 Install New Software... to open the install dialog. In this dialog, copy the update site of the version you want to install into the Work with: field and press Enter . If you want to install the latest released version, use this update site: https://artifacts.metaborg.org/content/unzip/releases-unzipped/org/metaborg/spoofax.lwb.eclipse.repository/0.22.0/spoofax.lwb.eclipse.repository-0.22.0.zip-unzip/ In the table below, check the checkbox next to Spoofax LWB and press Next > . A dialog will pop up with install details. Press Finish and wait for Eclipse to install the plugin. If a Security Warning pops up, press Install anyway , as Spoofax 3 is currently not signed. Finally, a dialog will pop up asking you to restart Eclipse. Press Restart Now to restart Eclipse. After restarting Eclipse, all projects must be cleaned in order to prevent compatibility issues. In the main menu of Eclipse, select Project \u2023 Clean... , check the Clean all project checkbox, and press Clean . Then, rebuild all projects with Project \u2023 Build All from the main menu, or by pressing Cmd + B on macOS or Ctrl + B on others. Downgrading \u00b6 Eclipse does not support directly downgrading to a previous version. Therefore, we must first uninstall the plugin. In the main menu, select Help \u2023 Install New Software... to open a new dialog. In this dialog, click the already installed link to open a list of all installed features. In that list, select Spoofax LWB and press Uninstall... . A dialog will pop up detailing the uninstall details. Press Finish and wait for Eclipse to uninstall the plugin. Finally, a dialog will pop up asking you to restart Eclipse. Press Restart Now to restart Eclipse. After Eclipse has restarted, follow the updating instructions above, but instead of using the latest update site, use the update site of an older version instead.","title":"Updating & Downgrading"},{"location":"guide/eclipse_lwb/update/#updating-downgrading","text":"","title":"Updating & Downgrading"},{"location":"guide/eclipse_lwb/update/#updating","text":"The Spoofax 3 language workbench Eclipse plugin can be updated as follows. In the main menu of Eclipse, select Help \u2023 Install New Software... to open the install dialog. In this dialog, copy the update site of the version you want to install into the Work with: field and press Enter . If you want to install the latest released version, use this update site: https://artifacts.metaborg.org/content/unzip/releases-unzipped/org/metaborg/spoofax.lwb.eclipse.repository/0.22.0/spoofax.lwb.eclipse.repository-0.22.0.zip-unzip/ In the table below, check the checkbox next to Spoofax LWB and press Next > . A dialog will pop up with install details. Press Finish and wait for Eclipse to install the plugin. If a Security Warning pops up, press Install anyway , as Spoofax 3 is currently not signed. Finally, a dialog will pop up asking you to restart Eclipse. Press Restart Now to restart Eclipse. After restarting Eclipse, all projects must be cleaned in order to prevent compatibility issues. In the main menu of Eclipse, select Project \u2023 Clean... , check the Clean all project checkbox, and press Clean . Then, rebuild all projects with Project \u2023 Build All from the main menu, or by pressing Cmd + B on macOS or Ctrl + B on others.","title":"Updating"},{"location":"guide/eclipse_lwb/update/#downgrading","text":"Eclipse does not support directly downgrading to a previous version. Therefore, we must first uninstall the plugin. In the main menu, select Help \u2023 Install New Software... to open a new dialog. In this dialog, click the already installed link to open a list of all installed features. In that list, select Spoofax LWB and press Uninstall... . A dialog will pop up detailing the uninstall details. Press Finish and wait for Eclipse to uninstall the plugin. Finally, a dialog will pop up asking you to restart Eclipse. Press Restart Now to restart Eclipse. After Eclipse has restarted, follow the updating instructions above, but instead of using the latest update site, use the update site of an older version instead.","title":"Downgrading"},{"location":"guide/static-semantics/code-completion/","text":"How to Enable Semantic Code Completion \u00b6 Semantic code completion is now part of Spoofax 3. Enabling Semantic Code Completion \u00b6 To enable support for semantic code completion in your language: Add the following to your language's spoofaxc.cfg file: tego-runtime {} code-completion {} Declare the following strategies in your language's main.str2 file, where MyLang is the name of your languages (as defined in spoofaxc.cfg : rules // Analysis downgrade-placeholders = downgrade-placeholders-MyLang upgrade-placeholders = upgrade-placeholders-MyLang is-inj = is-MyLang-inj-cons pp-partial = pp-partial-MyLang-string pre-analyze = explicate-injections-MyLang post-analyze = implicate-injections-MyLang In your Statix files, for each rule define a predicate that accepts a placeholder where a syntactic sort is permitted. For example: rules // placeholders programOk ( Module-Plhdr ()). declOk ( _ , Decl-Plhdr ()). typeOfExp ( _ , Exp-Plhdr ()) = _ . Using Semantic Code Completion \u00b6 In Eclipse, in a file of your language, type the placeholder somewhere where it is permitted. Unless overridden, the placeholder is a sort name within double square brackets, such as [[Exp]] . Then put the caret on the placeholder and press Cmd + Space on macOS (or Ctrl + Space on Linux/Windows) to invoke code completion. In a future release the placeholder will not need to be input explicitly. Limitations \u00b6 For this first release of semantic code completion, there are some limitations: You have to type the placeholder explicitly to invoke code completion. If the file contains errors, code completion might fail to return results. If you have catch-all predicates, code completion will not work. For example: // This prevents code completion from finding completions: typeOfExp ( _ , _ ) = _ : - try { false } | warning $[ This expression is not yet implemented ] . These limitations will be lifted in subsequent releases.","title":"Semantic Code Completion"},{"location":"guide/static-semantics/code-completion/#how-to-enable-semantic-code-completion","text":"Semantic code completion is now part of Spoofax 3.","title":"How to Enable Semantic Code Completion"},{"location":"guide/static-semantics/code-completion/#enabling-semantic-code-completion","text":"To enable support for semantic code completion in your language: Add the following to your language's spoofaxc.cfg file: tego-runtime {} code-completion {} Declare the following strategies in your language's main.str2 file, where MyLang is the name of your languages (as defined in spoofaxc.cfg : rules // Analysis downgrade-placeholders = downgrade-placeholders-MyLang upgrade-placeholders = upgrade-placeholders-MyLang is-inj = is-MyLang-inj-cons pp-partial = pp-partial-MyLang-string pre-analyze = explicate-injections-MyLang post-analyze = implicate-injections-MyLang In your Statix files, for each rule define a predicate that accepts a placeholder where a syntactic sort is permitted. For example: rules // placeholders programOk ( Module-Plhdr ()). declOk ( _ , Decl-Plhdr ()). typeOfExp ( _ , Exp-Plhdr ()) = _ .","title":"Enabling Semantic Code Completion"},{"location":"guide/static-semantics/code-completion/#using-semantic-code-completion","text":"In Eclipse, in a file of your language, type the placeholder somewhere where it is permitted. Unless overridden, the placeholder is a sort name within double square brackets, such as [[Exp]] . Then put the caret on the placeholder and press Cmd + Space on macOS (or Ctrl + Space on Linux/Windows) to invoke code completion. In a future release the placeholder will not need to be input explicitly.","title":"Using Semantic Code Completion"},{"location":"guide/static-semantics/code-completion/#limitations","text":"For this first release of semantic code completion, there are some limitations: You have to type the placeholder explicitly to invoke code completion. If the file contains errors, code completion might fail to return results. If you have catch-all predicates, code completion will not work. For example: // This prevents code completion from finding completions: typeOfExp ( _ , _ ) = _ : - try { false } | warning $[ This expression is not yet implemented ] . These limitations will be lifted in subsequent releases.","title":"Limitations"},{"location":"include/download/","text":"","title":"Download"},{"location":"reference/anatomy_language_implementation/","text":"Anatomy of a language implementation \u00b6 In this section we give a high-level overview of what a Spoofax 3 language implementation is, dive into details, and explain how such a implementation can be manually written or completely generated from a high-level language specification Overview \u00b6 In essence, a language implementation in Spoofax 3 is nothing more than a standard Java library (e.g., a JAR file) with Java classes implementing or delegating to the various functionalities of the language such as parsing and transformations, as well as bundled resources such as a parse table which is loaded and interpreted at runtime. Therefore, Spoofax 3 language implementations are very easy to use in the Java ecosystem by just distributing the JAR file of the language, or by publishing/consuming it as a library with a build system such as Gradle. Furthermore, since no classloading or class generation is used, GraalVM native image can be used to ahead-of-time compile your language implementation into native code which does not require a JVM at all, and significantly reduces the startup time of your language. Diving deeper, a language implementation is actually split into three parts: a language project that contains the base functionality of the language, an adapter project that adapts the language project to the interface of Spoofax, and platform projects that plug the adapter project into various other platforms such as a command-line interface (CLI) and Eclipse plugin (TODO: more details on supported platforms in a separate section). We will first explain these projects and why this separation was chosen. TODO: diagram? Language Project \u00b6 A language project contains the base functionality of a language, such as a parser, syntax highlighter, analyzer, and compiler for the language. Such a project is unstructured : it does not have to adhere to any interface or data format. Therefore, it may use any tooling, libraries, and data structures to implement the base functionality. This facilitates integration of existing tools and minimal dependencies. A language project is just a Java library and can thus be used in a standalone fashion. However, there is no glue between base functionality, requiring manual implementation of a parse-analyze-compile pipeline for example. Furthermore, because the project is unstructured, we cannot provide any integration with other platforms such as a CLI and Eclipse plugin. Therefore, using a language project as a standalone library is a bit of a niche use case for when minimal dependencies or full control is absolutely necessary. Because it is such a niche use case, the default in Spoofax 3 is to merge it together with the adapter project. In essence, an adapter project adapts a language project to Spoofax 3. To understand why, we first explain the high-level architecture of Spoofax 3. Spoofax 3 architecture overview \u00b6 Spoofax 3 provides a general interface for language implementations: LanguageInstance , which is used by platforms to automatically plug languages into their platform. For example, LanguageInstance has functionality for syntax highlighting , which when given a resource of the language, returns a syntax highlighting for that resource. (TODO: more details on the functionality in LanguageInstance in a separate section) Furthermore, Spoofax 3 uses PIE ; a framework for building incremental pipelines, build systems, and compilers; to incrementalize the language implementation. Instead of directly computing the syntax highlighting for a resource, we create a task that returns the syntax highlighting when demanded, with PIE taking care of whether it should recompute the syntax highlighting because the resource (or the syntax highlighting implementation) changed, or if it can just be returned from a cache. (TODO: more details on PIE in a separate section) A platform such as Eclipse or IntelliJ can then take a LanguageInstance implementation, demand the syntax highlighting task, and show the result it in the editor for your language. Therefore, any language that implements LanguageInstance can get syntax highlighting in Eclipse, IntelliJ, and any other supported platforms for free, with PIE taking care of coarse-grained incrementalization. To receive the benefits of Spoofax 3, the adapter project must thus be implemented for your language. Adapter Project \u00b6 An adapter project implements Spoofax 3's LanguageInstance using the language project. This requires glue code between the unstructured language project and the structured LanguageInstance interface. For example, you would need to convert the data structure that the syntax highlighter of your language returns, to one that Spoofax 3 understands: the Styling class. Furthermore, because Spoofax 3 uses PIE, we also need to implement a PIE task definition that implements the (re)computing of syntax highlighting, as well as mark all dependencies that should cause the syntax highlighting to be recomputed. We also need to be able to instantiate your implementation of LanguageInstance . In case this is non-trivial, the recommended practice is to use dependency injection to achieve proper separation of concerns. A dependency injection framework such as Dagger is recommended (we use it extensively in Spoofax 3) because it catches dependency injection errors at compile-time, and does not require runtime class loading or generation. This may sound like you would need to write a lot of boilerplate. However, we provide a compiler that generates all this boilerplate for you. It is only necessary to write this boilerplate if you are integrating existing tooling. Even then, the compiler can generate some of the boilerplate for you. More details on the compiler can be found in the developing language implementations section. Platform Projects \u00b6 TODO: every language-platform combination is a separate project to support ahead-of-time compilation, static loading, and customization of the platform project. CLI: can be ahead-of-time compiled with GraalVM native image to create a native Windows/macOS/Linux CLI for your language Eclipse/IntelliJ: statically loaded plugin that can be deployed with Eclipse/IntelliJ, and can be fully customized. Developing Language Implementations \u00b6 So far we have talked about what a language implementation is, but not yet how one is developed, which we will dive into now. Language implementations are by default fully generated from a high-level language specification using the Spoofax 3 compiler, thereby supporting iterative language development with low boilerplate. However, it is possible implement parts of or even the entire language/adapter project by hand, facilitating the integration of existing tools and languages. TODO: dynamic loading","title":"Anatomy of a language implementation"},{"location":"reference/anatomy_language_implementation/#anatomy-of-a-language-implementation","text":"In this section we give a high-level overview of what a Spoofax 3 language implementation is, dive into details, and explain how such a implementation can be manually written or completely generated from a high-level language specification","title":"Anatomy of a language implementation"},{"location":"reference/anatomy_language_implementation/#overview","text":"In essence, a language implementation in Spoofax 3 is nothing more than a standard Java library (e.g., a JAR file) with Java classes implementing or delegating to the various functionalities of the language such as parsing and transformations, as well as bundled resources such as a parse table which is loaded and interpreted at runtime. Therefore, Spoofax 3 language implementations are very easy to use in the Java ecosystem by just distributing the JAR file of the language, or by publishing/consuming it as a library with a build system such as Gradle. Furthermore, since no classloading or class generation is used, GraalVM native image can be used to ahead-of-time compile your language implementation into native code which does not require a JVM at all, and significantly reduces the startup time of your language. Diving deeper, a language implementation is actually split into three parts: a language project that contains the base functionality of the language, an adapter project that adapts the language project to the interface of Spoofax, and platform projects that plug the adapter project into various other platforms such as a command-line interface (CLI) and Eclipse plugin (TODO: more details on supported platforms in a separate section). We will first explain these projects and why this separation was chosen. TODO: diagram?","title":"Overview"},{"location":"reference/anatomy_language_implementation/#language-project","text":"A language project contains the base functionality of a language, such as a parser, syntax highlighter, analyzer, and compiler for the language. Such a project is unstructured : it does not have to adhere to any interface or data format. Therefore, it may use any tooling, libraries, and data structures to implement the base functionality. This facilitates integration of existing tools and minimal dependencies. A language project is just a Java library and can thus be used in a standalone fashion. However, there is no glue between base functionality, requiring manual implementation of a parse-analyze-compile pipeline for example. Furthermore, because the project is unstructured, we cannot provide any integration with other platforms such as a CLI and Eclipse plugin. Therefore, using a language project as a standalone library is a bit of a niche use case for when minimal dependencies or full control is absolutely necessary. Because it is such a niche use case, the default in Spoofax 3 is to merge it together with the adapter project. In essence, an adapter project adapts a language project to Spoofax 3. To understand why, we first explain the high-level architecture of Spoofax 3.","title":"Language Project"},{"location":"reference/anatomy_language_implementation/#spoofax-3-architecture-overview","text":"Spoofax 3 provides a general interface for language implementations: LanguageInstance , which is used by platforms to automatically plug languages into their platform. For example, LanguageInstance has functionality for syntax highlighting , which when given a resource of the language, returns a syntax highlighting for that resource. (TODO: more details on the functionality in LanguageInstance in a separate section) Furthermore, Spoofax 3 uses PIE ; a framework for building incremental pipelines, build systems, and compilers; to incrementalize the language implementation. Instead of directly computing the syntax highlighting for a resource, we create a task that returns the syntax highlighting when demanded, with PIE taking care of whether it should recompute the syntax highlighting because the resource (or the syntax highlighting implementation) changed, or if it can just be returned from a cache. (TODO: more details on PIE in a separate section) A platform such as Eclipse or IntelliJ can then take a LanguageInstance implementation, demand the syntax highlighting task, and show the result it in the editor for your language. Therefore, any language that implements LanguageInstance can get syntax highlighting in Eclipse, IntelliJ, and any other supported platforms for free, with PIE taking care of coarse-grained incrementalization. To receive the benefits of Spoofax 3, the adapter project must thus be implemented for your language.","title":"Spoofax 3 architecture overview"},{"location":"reference/anatomy_language_implementation/#adapter-project","text":"An adapter project implements Spoofax 3's LanguageInstance using the language project. This requires glue code between the unstructured language project and the structured LanguageInstance interface. For example, you would need to convert the data structure that the syntax highlighter of your language returns, to one that Spoofax 3 understands: the Styling class. Furthermore, because Spoofax 3 uses PIE, we also need to implement a PIE task definition that implements the (re)computing of syntax highlighting, as well as mark all dependencies that should cause the syntax highlighting to be recomputed. We also need to be able to instantiate your implementation of LanguageInstance . In case this is non-trivial, the recommended practice is to use dependency injection to achieve proper separation of concerns. A dependency injection framework such as Dagger is recommended (we use it extensively in Spoofax 3) because it catches dependency injection errors at compile-time, and does not require runtime class loading or generation. This may sound like you would need to write a lot of boilerplate. However, we provide a compiler that generates all this boilerplate for you. It is only necessary to write this boilerplate if you are integrating existing tooling. Even then, the compiler can generate some of the boilerplate for you. More details on the compiler can be found in the developing language implementations section.","title":"Adapter Project"},{"location":"reference/anatomy_language_implementation/#platform-projects","text":"TODO: every language-platform combination is a separate project to support ahead-of-time compilation, static loading, and customization of the platform project. CLI: can be ahead-of-time compiled with GraalVM native image to create a native Windows/macOS/Linux CLI for your language Eclipse/IntelliJ: statically loaded plugin that can be deployed with Eclipse/IntelliJ, and can be fully customized.","title":"Platform Projects"},{"location":"reference/anatomy_language_implementation/#developing-language-implementations","text":"So far we have talked about what a language implementation is, but not yet how one is developed, which we will dive into now. Language implementations are by default fully generated from a high-level language specification using the Spoofax 3 compiler, thereby supporting iterative language development with low boilerplate. However, it is possible implement parts of or even the entire language/adapter project by hand, facilitating the integration of existing tools and languages. TODO: dynamic loading","title":"Developing Language Implementations"},{"location":"reference/configuration/","text":"Configuration \u00b6 The main entry point of a language definition is the spoofaxc.cfg (Spoofax compiler configuration) file, written in the CFG language. The goal of this config file is to configure basic options, enable/disable features, point to main source files of meta-languages, to add/override behaviour, and to serve as an anchor on the filesystem. The directory that the spoofaxc.cfg file is in is called the \"root directory\" of the language definition, and any relative paths are resolved relative to that directory. The CFG language has domain-specific syntax for configuring language definitions. However, the syntax follows these conventions: Options are assigned a value with $Option = $Expression . Unless specified otherwise, options may only be given once. Sections $Section { ... } may enable features and group options. Lists $List [ ..., ... ] indicate an option/section may be given 0-many times. Let bindings let $Name = $Expression can be used to give values a name that can be (re-)used in the rest of the configuration file. If something in the documentation is unclear, the CFG language definition can be found here . In its most basic form, the spoofaxc.cfg file for a language named Calc looks as follows: name = \"Calc\" which assigns the string \"Calc\" to the name option. A more interesting example configures more options and enables syntax definition: name = \"Calc\" java-class-id-prefix = java Calc file-extension = \"calc\" sdf3 {} parser { default-start-symbol = sort Program } Here, java Calc is assigned to the java-class-id-prefix option. The sdf3 {} section from the example is empty, but is used to enable the SDF3 meta-language. The parser section enables generation of a parser, and also sets the default start symbol to use to sort Program . Literals \u00b6 Literals are expressions that are usually directly assigned to options, or bound to a name with let bindings. CFG has the following literals: Syntax Example(s) Type (true|false) true false Boolean -?[0-9]+ 1 -20 Integer [0-9]+ 1 20 Unsigned integer '(~[\\'\\$\\n\\r\\\\] | \\\\~[\\n\\r])' '[' 'l' Character \"(~[\\\"\\$\\n\\r\\\\] | \\\\~[\\n\\r])*\" \"foo\" \"bar\" String (./|/)~[\\n\\r\\,\\;\\]\\)\\}\\ ]* ./relative/file /absolute/file Filesystem path $JavaIdChars Java foo Java identifier $JavaQIdLit Java foo.bar.Baz Qualified Java identifier task-def $JavaQIdLit task-def foo.bar.Baz Qualified Java identifier that represents a task definition sort [a-zA-Z0-9\\-\\_\\$]+ sort Start SDF3 sort identifier strategy [a-zA-Z0-9\\-\\_\\$]+ strategy Start Stratego strategy identifier $CoordinateChars:$CoordinateChars:$CoordinateChars org.metaborg:strategolib:1.0.0 Coordinate $CoordinateChars:$CoordinateChars:$CoordinateRequirementChars org.metaborg:strategolib:* Coordinate requirement With the following syntax non-terminals: Name Syntax JavaIdChars [a-zA-Z\\_\\$][a-zA-Z0-9\\_\\$]* JavaQIdLit $JavaIdChars(\\.$JavaIdChars)* CoordinateChars (~[\\\"\\:\\;\\,\\*\\$\\{\\}\\[\\]\\n\\r\\\\\\ ]|\\\\~[\\n\\r]) CoordinateRequirementChars ($CoordinateChars|\\*) For Java, SDF3 sort, and Stratego strategy identifiers, the corresponding keywords of those languages are rejected as identifiers. Let bindings \u00b6 Let bindings of the form let $Name = $Expression bind a name to an expression, for example: let showParsedAst = task-def mb.helloworld.task.HelloWorldShowParsedAst let showParsedAstCommand = command-def { task-def = showParsedAst ... } editor-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand ... } ] ] creates a binding from name showParsedAst to task-def mb.helloworld.task.HelloWorldShowParsedAst , which we then pass to the task-def option in command-def . The command in turn is bound to showParsedAstCommand , assigned to the command-def option in a command-action section. Top-level options \u00b6 The following top-level options exist: Syntax Required? Description Type group = $Expression no Group identifier of the language, used as the group / groupId in the Java ecosystem. Defaults to org.metaborg . String id = $Expression no Artifact identifier of the language, used as the name / artifactId in the Java ecosystem. Defaults to the name of the language uncapitalized. String name = $Expression yes Name of the language. String version = $Expression no Version of the language, used as the version in the Java ecosystem. Defaults to 0.1.0 . String file-extension = $Expression no File extension of the language. May be given multiple times. Defaults to the name of the language transformed to fit in 3 characters. String java-package-id-prefix = $Expression no The prefix to add before all package identifiers in Java source files. Defaults to mb.$Name where $Name is transformed to be a valid package identifier. Qualified Java identifier java-class-id-prefix = $Expression no The prefix to add before all Java classes. Defaults to the name of the language transformed to be a valid class identifier. Java identifier source-directory = $Expression no Path relative to the root directory that has the sources of the language definition. Defaults to src . Path build-directory = $Expression no Path relative to the root directory that has the generated sources and build outputs when building the language definition. Defaults to build . Path Commands \u00b6 Commands are sections that are also expressions, typically assigned to a name with a let binding, with the following form: let $Name = command-def { $CommandOption* } The following options are available in a command: Syntax Required? Description Type task-def = $Expression yes The task definition that the command will execute. Qualified Java identifier that represents a task definition, or qualified Java identifier type = $Expression no The fully qualified Java type we want this command to be generated as. Can be omitted to generate a type based on the name of the task definition. Qualified Java identifier display-name = $Expression yes The display name of the command. String description = $Expression no The optional description of the command. String supported-execution-types = [($ExecutionType ,)*] no The optional supported execution types of the command. Defaults to [Once, Continuous] . n/a args-type = $Expression no The fully qualified Java type of the argument class. Can be omitted if the argument class is a nested class named Args of the task definition. Qualified Java identifier parameters = [ $Parameter* ] yes The description of the parameters of the command n/a The following ExecutionType s are supported: Once indicates a that this command supports being executed as a one-shot command. Continuous indicates that this command supports being executed every time the source file changes. A $Parameter has the form $Identifier { $ParameterOptions } with the following options: Syntax Required? Description Type type = $Expression yes The fully qualified Java of the type of the parameter. This must match the type of the parameter inside the args-type of the command. Qualified Java identifier required = $Expression no Whether the parameter is required. Defaults to true . Boolean converter-type = $Expression no The argument converter for this parameter, which can convert a String value to the type of this parameter. Must implement the ArgConverter interface. Qualified Java identifier argument-providers = [($ArgumentProvider ,)*] no Argument providers for this parameter that attempt to automatically provide a fitting argument. When providing an argument fails, the next argument provider in the list will be attempted. If no arguments can be provided, and the argument is required, then the argument must be provided by the user that executes the command, or executing the command will fail. n/a The following ArgumentProvider s are supported: Value($Expression) provides a default value given by the expression. The expression must match the type of the parameter, even though this is not currently checked. Context($CommandContext) attempts to infer the argument by context. The following CommandContext s are supported: Directory : attempt to infer a ResourcePath to an existing directory. For example, when right-clicking a directory in an IDE to execute a command on that directory. File : attempt to infer a ResourcePath to an existing file. For example, when right-clicking a file in an IDE to execute a command on that directory, or when executing a command in an editor for a file. HierarchicalResource : attempt to infer a ResourcePath to a hierarchical resource. A hierarchical resource is a resource that belongs to a (tree) hierarchy, such as a file or directory on the local filesystem. Use this when the command relies on the resource being in a filesystem, but does not care whether it is a directory or a file. ReadableResource : attempt to infer a ResourceKey to a readable resource. This is more general than File , as we only ask for a resource that can be read, not one that belongs to a (local) filesystem. Use this when the command does not rely on the resource being in a filesystem. Region : attempt to infer a Region in a source file. Inference succeeds when the context has a selection of size 1 or larger. For example, when executing a command in an editor that has a selection, the region will be that selection. Offset : attempt to infer an int representing an offset in a source file. Inference succeeds when the context has a cursor offset (i.e., a selection of size 0 or larger). For example, when executing a command in an editor, the offset will be the offset to the cursor in the editor. EnclosingContext($EnclosingCommandContext) attempts to infer the argument by the enclosing context. The following EnclosingCommandContext s are supported: Project : attempt to infer a ResourcePath to the enclosing project. For example, when executing a command in an IDE on a file, directory, or editor for a file, that belongs to a project. Or when executing a command in a CLI, the directory will be the current working directory. Directory : attempt to infer a ResourcePath to the enclosing directory. For example, when executing a command in the context of a file, directory, or editor for a file, the directory will be the parent of that file/directory. Here is an example of a command that shows the parsed AST by taking one file argument that is inferred from context: let showParsedAstCommand = command-def { type = java mb.helloworld.command.HelloWorldShowParsedAstCommand task-def = showParsedAst args-type = java mb.helloworld.task.HelloWorldShowParsedAst.Args display-name = \"Show parsed AST\" description = \"Shows the parsed AST\" supported-execution-types = [Once, Continuous] parameters = [ file = parameter { type = java mb.resource.ResourceKey required = true argument-providers = [Context(ResourceKey)] } ] } Menu items \u00b6 Menu items take the form of: a separator representing a horizontal line in a menu used to separate groups of menu items. a menu $Expression [ $MenuItem* ] representing a (nested) menu with a display name defined by the expression which must be a string, and a list of nested menu items. a command-action { $CommandActionOption* } representing an action that executes a command when a user clicks on it. A command action has the following options: Syntax Required? Description Type command-def = $Expression yes The command to execute. Command or qualified Java identifier execution-type = $ExecutionType yes How the command should be executed. n/a required-resource-types = [($ResourceType ,)*] no On which kinds of resources this menu item will be shown on resource context menus. Defaults to empty. If empty, it will not be hidden based on resources. n/a required-enclosing-resource-types = [($EnclosingResourceType ,)*] no On which kinds of enclosing resources this menu item will be shown on resource context menus. Defaults to empty. If empty, it will not be hidden based on enclosing resource. n/a required-editor-file-types = [($EditorFileType ,)*] no On which kinds of editors belonging to certain file types this menu item will be shown. Defaults to empty. If empty, it will not be hidden based on editor file types. n/a required-editor-selection-types = [($EditorSelectionType ,)*] no On which kinds of editor selection types this menu item will be shown. Defaults to empty. If empty, it will not be hidden based on editor selections. n/a display-name = $Expression no The display name of the command action. Defaults to the display name of the command String description = $Expression no The description of he command action. Defaults to the description of the command String The following ResourceType s are supported: Directory : the menu item will only be shown when a directory is selected. File : the menu item will only be shown when a file is selected. The following EnclosingResourceType s are supported: Project : the menu item will only be shown when the selected resource has an enclosing project. Directory: : the menu item will only be shown when the selected resource has an enclosing directory. The following EditorFileType s are supported: HierarchicalResource : the menu item will only be shown when the editor belongs to a hierarchical resource. That is, a resource that belongs to a tree, such as a file or directory on the local filesystem. ReadableResource : the menu item will only be shown when the editor belongs to a readable resource. The following EditorSelectionType s are supported: Region : the menu item will only be shown when a region with size >0 in the source file is selected. Offset : the menu item will only be shown in the context of an editor with a cursor. Menus \u00b6 Menu items are assigned to 3 particular menus: editor-context-menu [ $MenuItem* ] : the context menu that gets shown in editors of the language, for example when right-clicking in an editor of the language in an IDE. Spoofax automatically creates a top-level submenu with the name of the language to host the editor context menu items. The required-editor-file-types and required-editor-selection-types options are used to filter menu items. resource-context-menu [ $MenuItem* ] : the context menu that gets shown in resource explorers, for example when right-clicking in the file browser in an IDE. Spoofax automatically creates a top-level submenu with the name of the language to host the editor context menu items. The required-resource-types and required-enclosing-resource-types options are used to filter menu items. main-menu [ $MenuIitem* ] : the main menu of the language, which is shown on the menu bar in IDEs. When no main-menu section is given, it defaults to the same menu as editor-context-menu . For example, we can assign the command defined earlier to several menus: editor-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand execution-type = Once } command-action { command-def = showParsedAstCommand execution-type = Continuous } ] ] resource-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand execution-type = Once required-resource-types = [File] } ] ] Language feature sections \u00b6 Parser \u00b6 The parser { $ParserOption* } section enables generation of a parser, and groups options. The sdf3 section must be enabled when the parser section is enabled. The following ParserOption s are supported: Syntax Required? Description Type default-start-symbol = $Expression yes The start symbol to use when no specific one is provided. SDF3 sort identifier variant = $ParserVariant no The parser variant to use. Defaults to jsglr1 . n/a The following ParserVariant s are supported: jsglr1 : uses the JSGLR1 parser. jsglr2 { $Jsglr2Option* } : uses the JSGLR2 parser. The following Jsglr2Option s are supported: preset = $Jsglr2Preset : sets the JSGLR2 preset to use. The following Jsglr2Preset s are supported: Standard Elkhound Recovery RecoveryElkhound DataDependent LayoutSensitive Composite Incremental IncrementalRecovery Comment symbols \u00b6 The comment-symbols { $CommentSymbolOption* } section enables specification of line and block comment characters, which are required for the \"toggle comment\" editor service. The following CommentSymbolOption s are supported: Syntax Required? Description Type line = $Expression no Adds a line comment symbol. Can be given multiple times to list multiple line comment symbols. The first one will be used to comment a line with the \"toggle comment\" editor service. String block = $Expression * $Expression no Adds block comment symbols, with an opening and close symbol. Current \"toggle comment\" editor services do not use block comment symbols yet. String Bracket symbols \u00b6 The bracket-symbols { $BracketSymbolOption* } section enables specification of bracket symbols (e.g., square brackets, curly brackets, parentheses, etc.), which are required for the \"bracket matching\" editor service. The following BracketSymbolOption s are supported: Syntax Required? Description Type bracket = $Expression * $Expression no Adds bracket symbols, with an opening and closing symbol. Can be given multiple times to list multiple bracket symbols. Character Styler \u00b6 The styler { $StylerOption* } section enables generation of a styler, and groups options. The esv section must be enabled when the styler section is enabled. Currently, no StylerOption s are supported. Constraint analyzer \u00b6 The constraint-analyzer { $ConstraintAnalyzerOption* } section enables generation of a constraint analyzer, and groups options. The statix section must be enabled when the constraint-analyzer section is enabled. The following ConstraintAnalyzerOption s are supported: Syntax Required? Description Type multi-file = $Expression no Whether multi-file analysis is enabled. Defaults to false . Boolean stratego-strategy = $Expression no The stratego strategy entry-point that handles communication with the constraint-solver. Defaults to editor-analyze . Stratego strategy identifier default-statix-message-stacktrace-length = $Expression no The default Statix message stacktrace length to use. Default is implementation-defined. Does nothing if Statix is not enabled. Unsigned integer default-statix-message-term-depth = $Expression no The default Statix message term depth to use. Default is implementation-defined. Does nothing if Statix is not enabled. Unsigned integer default-statix-test-log-level = $Expression no The default Statix test log level to use. Default is implementation-defined. Does nothing if Statix is not enabled. String default-statix-supress-cascading-errors = $Expression no Whether to suppress cascading errors by default. Default is implementation-defined. Does nothing if Statix is not enabled. Boolean statix-solver-mode = $StatixSolverMode no Statix solver mode. Defaults to traditional . Does nothing if Statix is not enabled. n/a The following StatixSolverMode s are supported: traditional concurrent incremental Multi-language analyzer \u00b6 The multilang-analyzer { $MultilangAnalyzerOption* } section enables generation of a multi-language analyzer, and groups options. The constraint-analyzer and statix sections must be enabled when the multilang-analyzer section is enabled. Currently, no MultilangAnalyzerOption s are supported. Stratego runtime \u00b6 The stratego-runtime { $StrategoRuntimeOption* } section enables generation of a stratego runtime, and groups options. The stratego section must be enabled when the stratego-runtime section is enabled. The following StrategoRuntimeOption s are supported: Syntax Required? Description Type strategy-package-id = $Expression no Adds a package as a private Stratego package. Can be specified multiple times. Java package identifier interop-registerer-by-reflection = $Expression no Adds an interop registerer to load by reflection. Can be specified multiple times. Java type identifier add-spoofax2-primitives = $Expression no Whether to add the Spoofax 2 Stratego primitives. Boolean add-nabl2-primitives = $Expression no Whether to add the NaBL2 Stratego primitives. Boolean add-statix-primitives = $Expression no Whether to add the Statix Stratego primitives. Boolean with-primitive-library = $Expression no Adds a Stratego primitive strategies library (implementing org.spoofax.interpreter.library.IOperatorRegistry ) to the generated StrategoRuntimeBuilderFactory . The library must have an @Inject constructor. Can be specified multiple times. Java type identifier with-interop-registerer = $Expression no Adds a Stratego interop registerer (implementing org.strategoxt.lang.InteropRegisterer ) to the generated StrategoRuntimeBuilderFactory . The registerer must have an @Inject constructor. Can be specified multiple times. Java type identifier class-kind = $Expression no Specifies whether the classes are generated ( Generated ) or provided manually ( Manual ). Defaults to Generated . Generated or Manual base-StrategoRuntimeBuilderFactory = $Expression no Package and name of the generated StrategoRuntimeBuilderFactory . Java type identifier extend-StrategoRuntimeBuilderFactory = $Expression no Package and name of the extending StrategoRuntimeBuilderFactory , if any. Java type identifier Completer \u00b6 The completer { $CompleterOption* } section enables generation of a code completer, and groups options. The constraint-analyzer and statix sections must be enabled when the completer section is enabled. Currently, no CompleterOption s are supported. Reference resolution \u00b6 The reference-resolution { $ReferenceResolutionOption* } section enables generation of the reference resolver editor service, and groups options. The following ReferenceResolutionOption s are supported: Syntax Required? Description Type variant = $ReferenceResolutionVariant yes The reference resolution variant to use. n/a The following ReferenceResolutionVariant s are supported: Stratego-based: stratego { strategy = strategy $Strategy } where Strategy is a Stratego strategy, typically editor-resolve . Hover tooltips \u00b6 The hover { $HoverOption* } section enables generation of the hover text editor service, and groups options. The following HoverOption s are supported: Syntax Required? Description Type variant = $HoverVariant yes The reference resolution variant to use. n/a The following HoverVariant s are supported: Stratego-based: stratego { strategy = strategy $Strategy } where Strategy is a Stratego strategy, typically editor-hover . Meta-language sections \u00b6 SDF3 \u00b6 The sdf3 { $Sdf3Option* } section enables syntax definition with SDF3 . The parser section must be enabled when the sdf3 section is enabled. The following Sdf3Option s are supported: Syntax Required? Description Type source = $Sdf3Source no The source of the SDF3 definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./start.sdf3 as its main file relative to the main source directory. n/a The following $Sdf3Source s are supported: Files: files { $Sdf3FilesOption* } Prebuilt: prebuilt { $Sdf3PrebuiltOption } The following Sdf3FilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main SDF3 file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main SDF3 file relative to the main-source-directory . Defaults to ./start.sdf3 . Path include-directory = $Expression no Adds an include directory from which to resolve SDF3 imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the SDF3 files in it accessable to dependencies. May be given multiple times. Path parse-table-generator { $ParseTableGeneratorOption* } no Parse table generator options. n/a placeholders { $PlaceholderOption* } no Placeholder syntax used in code completion. n/a stratego-concrete-syntax-extension-main-file = $Expression no Sets the main SDF3 file used to create a concrete syntax extension parse table. Path The following $Sdf3PrebuiltOption s are supported: Syntax Required? Description Type parse-table-aterm-file = $Expression yes The prebuilt SDF3 parse table ATerm file to use (usually called sdf.tbl ) relative to the root directory Path parse-table-persisted-file = $Expression yes The prebuilt SDF3 parse table persisted file to use (usually called sdf.bin ) relative to the root directory Path The following ParseTableGeneratorOption s are supported: Syntax Required? Description Type dynamic = $Expression no Whether the generated parse table is dynamic. Defaults to false . Boolean data-dependent = $Expression no Whether the generated parse table is data-dependent. Defaults to false . Boolean layout-sensitive = $Expression no Whether the generated parse table is layout-sensitive. Defaults to false . Boolean solve-deep-conflicts = $Expression no Whether the parse table generator solves deep priority conflicts. Defaults to true . Boolean check-overlap = $Expression no Whether the parse table generator checks for overlap. Defaults to false . Boolean check-priorities = $Expression no Whether the parse table generator checks priorities. Defaults to false . Boolean The following PlaceholderOption s are supported: Syntax Required? Description Type prefix = $Expression no The prefix to a placholder. Defaults to [[ , or empty if the other is defined. String postfix = $Expression no The postfix to a placeholder. Defaults to ]] , or empty if the other is defined. String ESV \u00b6 The esv { $EsvOption* } section enables syntax-based styling definition with ESV . The styler section must be enabled when the esv section is enabled. The following EsvOption s are supported: Syntax Required? Description Type source = $EsvSource no The source of the ESV definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.esv as its main file relative to the main source directory. n/a The following $EsvSource s are supported: Files: files { $EsvFilesOption* } Prebuilt: prebuilt { $EsvPrebuiltOption } The following EsvFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main ESV file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main ESV file relative to the main-source-directory . Defaults to ./main.esv . Path include-directory = $Expression no Adds an include directory from which to resolve ESV imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the ESV files in it accessable to dependencies. May be given multiple times. Path The following $EsvPrebuiltOption s are supported: Syntax Required? Description Type file = $Expression yes The prebuilt ESV file to use relative to the root directory Path Statix \u00b6 The statix { $StatixOption* } section enables static semantics definition with Statix . The constraint-anaylzer section must be enabled when the statix section is enabled. The following StatixOption s are supported: Syntax Required? Description Type source = $StatixSource no The source of the Statix definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.stx as its main file relative to the main source directory. n/a The following $StatixSource s are supported: Files: files { $StatixFilesOption* } Prebuilt: prebuilt { $StatixPrebuiltOption } The following StatixFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main Statix file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main Statix file relative to the main-source-directory . Defaults to ./main.stx . Path include-directory = $Expression no Adds an include directory from which to resolve Statix imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the Statix files in it accessable to dependencies. May be given multiple times. Path sdf3-statix-signature-generation = $Expression no Whether SDF3 to Statix signature generation is enabled. When enabled, stratego { sdf3-statix-explication-generation = true } must also be enabled. Defaults to false . Boolean The following $StatixPrebuiltOption s are supported: Syntax Required? Description Type spec-aterm-directory = $Expression yes The prebuilt Statix spec ATerm directory to use relative to the root directory Path Stratego \u00b6 The stratego { $StrategoOption* } section enables definition of transformations with Stratego . The stratego-runtime section must be enabled when the stratego section is enabled. The following StrategoOption s are supported: Syntax Required? Description Type source = $StrategoSource no The source of the Statix definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.str2 as its main file relative to the main source directory. n/a output-java-package = $Expression no The Java package into which compiled Stratego Java files are generated. Defaults to the language's package, followed by .strategies . String The following $StrategoSource s are supported: Files: files { $StrategoFilesOption* } The following $StrategoFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main Stratego file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main Stratego file relative to the main-source-directory . Defaults to ./main.str2 . Path include-directory = $Expression no Adds an include directory from which to resolve Stratego imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the Stratego files in it accessable to dependencies. May be given multiple times. Path with-import-strategy-package-id = $Expression no Adds a Java package to import in the Java files that have been compiled from Stratego files. For example, \"mb.mylang.strategies\" adds import import mb.mylang.strategies.* . May be given multiple times. String sdf3-statix-explication-generation = $Expression no Whether SDF3 to Statix injection explication/implication generation is enabled. When enabled, statix { sdf3-statix-signature-generation = true } must also be enabled. Defaults to false . Boolean language-strategy-affix = $Expression no The affix that is used to make certain generated strategies unique to the language. This is used both as a prefix and suffix. Defaults to name of the language transformed to a Stratego strategy identifier. Stratego strategy identifier concrete-syntax-extension-parse-table = $Expression no Adds a Stratego concrete syntax extension parse table that can be used to parse Stratego files with. May be given multiple times. Path Dependencies \u00b6 The dependencies [ $Dependency* ] section allows specifying dependencies to other language (library) projects. A dependency consists of a dependency expression ( $Exp ) that specifies the source of the dependency. The following expressions are supported: $Coordinate : an exact coordinate to a project, such as org.metaborg:strategolib:1.0.0 . Coordinates are resolved to a statically loaded language, or a dynamically loaded language definition. $CoordinateRequirement : a coordinate to a project with an open version, such as org.metaborg:strategolib:* , resolved identically to $Coordinate except that the latest available version will be chosen. $Path : a relative path (relative to the directory spoofaxc.cfg is in) to a language definition. Dependencies can be of a certain kind ( $DependencyKind ): Build : a dependency that is resolved at build time , meaning when the language definition is built, allowing the use of sources and binaries of the project the dependency points to. Run : a dependency that is resolved at run time , meaning when the language is executed, allowing use of the classes and tasks of the project the dependency points to. Note: this kind of dependency has not yet been implemented. The following Dependency s are supported: Syntax Description $Exp A dependency to a project defined by the expression, available at Build and Run time. $Exp { $DependencyOption* } A dependency to project defined by the expression, with configuration options. The following DependencyOption s are supported: kinds = [ $DependencyKind* ] : set the kinds at which this dependency is resolved. Additionally, build time dependencies can be quickly defined with the build-dependencies [ $BuildDependency* ] section. The following BuildDependency s are supported: Syntax Description $Exp A dependency to project defined by the expression, available only at Build time. For example, the following configuration adds build dependencies to common libraries shipped with Spoofax 3: build-dependencies [ org.metaborg : strategolib:* org.metaborg : gpp:* org.metaborg : libspoofax2:* org.metaborg : libstatix:* ] spoofaxc.lock \u00b6 The spoofaxc.lock file, which resides next to the spoofaxc.cfg file, contains values for several options that have defaults derived from other options, in order to keep these derived values stable even when the options they are derived from are changed. For example, when no java-class-id-prefix option is set in spoofaxc.cfg , it will be derived from the name option with some changes to make it compatible as a Java identifier, and is stored under shared.defaultClassPrefix in the spoofaxc.lock file. When you change the name of your language, the stored value will be used, keeping the class prefix the same, making it possible to rename the language without having to rename all class files. Therefore, the spoofaxc.lock file should be checked in to source control, in order to have reproducible builds. If you do want to re-derive a default from other options, remove the option from the spoofaxc.lock file and rebuild the language. The value will be re-derived and stored in spoofaxc.lock , after which you need to check it into source control again.","title":"Configuration"},{"location":"reference/configuration/#configuration","text":"The main entry point of a language definition is the spoofaxc.cfg (Spoofax compiler configuration) file, written in the CFG language. The goal of this config file is to configure basic options, enable/disable features, point to main source files of meta-languages, to add/override behaviour, and to serve as an anchor on the filesystem. The directory that the spoofaxc.cfg file is in is called the \"root directory\" of the language definition, and any relative paths are resolved relative to that directory. The CFG language has domain-specific syntax for configuring language definitions. However, the syntax follows these conventions: Options are assigned a value with $Option = $Expression . Unless specified otherwise, options may only be given once. Sections $Section { ... } may enable features and group options. Lists $List [ ..., ... ] indicate an option/section may be given 0-many times. Let bindings let $Name = $Expression can be used to give values a name that can be (re-)used in the rest of the configuration file. If something in the documentation is unclear, the CFG language definition can be found here . In its most basic form, the spoofaxc.cfg file for a language named Calc looks as follows: name = \"Calc\" which assigns the string \"Calc\" to the name option. A more interesting example configures more options and enables syntax definition: name = \"Calc\" java-class-id-prefix = java Calc file-extension = \"calc\" sdf3 {} parser { default-start-symbol = sort Program } Here, java Calc is assigned to the java-class-id-prefix option. The sdf3 {} section from the example is empty, but is used to enable the SDF3 meta-language. The parser section enables generation of a parser, and also sets the default start symbol to use to sort Program .","title":"Configuration"},{"location":"reference/configuration/#literals","text":"Literals are expressions that are usually directly assigned to options, or bound to a name with let bindings. CFG has the following literals: Syntax Example(s) Type (true|false) true false Boolean -?[0-9]+ 1 -20 Integer [0-9]+ 1 20 Unsigned integer '(~[\\'\\$\\n\\r\\\\] | \\\\~[\\n\\r])' '[' 'l' Character \"(~[\\\"\\$\\n\\r\\\\] | \\\\~[\\n\\r])*\" \"foo\" \"bar\" String (./|/)~[\\n\\r\\,\\;\\]\\)\\}\\ ]* ./relative/file /absolute/file Filesystem path $JavaIdChars Java foo Java identifier $JavaQIdLit Java foo.bar.Baz Qualified Java identifier task-def $JavaQIdLit task-def foo.bar.Baz Qualified Java identifier that represents a task definition sort [a-zA-Z0-9\\-\\_\\$]+ sort Start SDF3 sort identifier strategy [a-zA-Z0-9\\-\\_\\$]+ strategy Start Stratego strategy identifier $CoordinateChars:$CoordinateChars:$CoordinateChars org.metaborg:strategolib:1.0.0 Coordinate $CoordinateChars:$CoordinateChars:$CoordinateRequirementChars org.metaborg:strategolib:* Coordinate requirement With the following syntax non-terminals: Name Syntax JavaIdChars [a-zA-Z\\_\\$][a-zA-Z0-9\\_\\$]* JavaQIdLit $JavaIdChars(\\.$JavaIdChars)* CoordinateChars (~[\\\"\\:\\;\\,\\*\\$\\{\\}\\[\\]\\n\\r\\\\\\ ]|\\\\~[\\n\\r]) CoordinateRequirementChars ($CoordinateChars|\\*) For Java, SDF3 sort, and Stratego strategy identifiers, the corresponding keywords of those languages are rejected as identifiers.","title":"Literals"},{"location":"reference/configuration/#let-bindings","text":"Let bindings of the form let $Name = $Expression bind a name to an expression, for example: let showParsedAst = task-def mb.helloworld.task.HelloWorldShowParsedAst let showParsedAstCommand = command-def { task-def = showParsedAst ... } editor-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand ... } ] ] creates a binding from name showParsedAst to task-def mb.helloworld.task.HelloWorldShowParsedAst , which we then pass to the task-def option in command-def . The command in turn is bound to showParsedAstCommand , assigned to the command-def option in a command-action section.","title":"Let bindings"},{"location":"reference/configuration/#top-level-options","text":"The following top-level options exist: Syntax Required? Description Type group = $Expression no Group identifier of the language, used as the group / groupId in the Java ecosystem. Defaults to org.metaborg . String id = $Expression no Artifact identifier of the language, used as the name / artifactId in the Java ecosystem. Defaults to the name of the language uncapitalized. String name = $Expression yes Name of the language. String version = $Expression no Version of the language, used as the version in the Java ecosystem. Defaults to 0.1.0 . String file-extension = $Expression no File extension of the language. May be given multiple times. Defaults to the name of the language transformed to fit in 3 characters. String java-package-id-prefix = $Expression no The prefix to add before all package identifiers in Java source files. Defaults to mb.$Name where $Name is transformed to be a valid package identifier. Qualified Java identifier java-class-id-prefix = $Expression no The prefix to add before all Java classes. Defaults to the name of the language transformed to be a valid class identifier. Java identifier source-directory = $Expression no Path relative to the root directory that has the sources of the language definition. Defaults to src . Path build-directory = $Expression no Path relative to the root directory that has the generated sources and build outputs when building the language definition. Defaults to build . Path","title":"Top-level options"},{"location":"reference/configuration/#commands","text":"Commands are sections that are also expressions, typically assigned to a name with a let binding, with the following form: let $Name = command-def { $CommandOption* } The following options are available in a command: Syntax Required? Description Type task-def = $Expression yes The task definition that the command will execute. Qualified Java identifier that represents a task definition, or qualified Java identifier type = $Expression no The fully qualified Java type we want this command to be generated as. Can be omitted to generate a type based on the name of the task definition. Qualified Java identifier display-name = $Expression yes The display name of the command. String description = $Expression no The optional description of the command. String supported-execution-types = [($ExecutionType ,)*] no The optional supported execution types of the command. Defaults to [Once, Continuous] . n/a args-type = $Expression no The fully qualified Java type of the argument class. Can be omitted if the argument class is a nested class named Args of the task definition. Qualified Java identifier parameters = [ $Parameter* ] yes The description of the parameters of the command n/a The following ExecutionType s are supported: Once indicates a that this command supports being executed as a one-shot command. Continuous indicates that this command supports being executed every time the source file changes. A $Parameter has the form $Identifier { $ParameterOptions } with the following options: Syntax Required? Description Type type = $Expression yes The fully qualified Java of the type of the parameter. This must match the type of the parameter inside the args-type of the command. Qualified Java identifier required = $Expression no Whether the parameter is required. Defaults to true . Boolean converter-type = $Expression no The argument converter for this parameter, which can convert a String value to the type of this parameter. Must implement the ArgConverter interface. Qualified Java identifier argument-providers = [($ArgumentProvider ,)*] no Argument providers for this parameter that attempt to automatically provide a fitting argument. When providing an argument fails, the next argument provider in the list will be attempted. If no arguments can be provided, and the argument is required, then the argument must be provided by the user that executes the command, or executing the command will fail. n/a The following ArgumentProvider s are supported: Value($Expression) provides a default value given by the expression. The expression must match the type of the parameter, even though this is not currently checked. Context($CommandContext) attempts to infer the argument by context. The following CommandContext s are supported: Directory : attempt to infer a ResourcePath to an existing directory. For example, when right-clicking a directory in an IDE to execute a command on that directory. File : attempt to infer a ResourcePath to an existing file. For example, when right-clicking a file in an IDE to execute a command on that directory, or when executing a command in an editor for a file. HierarchicalResource : attempt to infer a ResourcePath to a hierarchical resource. A hierarchical resource is a resource that belongs to a (tree) hierarchy, such as a file or directory on the local filesystem. Use this when the command relies on the resource being in a filesystem, but does not care whether it is a directory or a file. ReadableResource : attempt to infer a ResourceKey to a readable resource. This is more general than File , as we only ask for a resource that can be read, not one that belongs to a (local) filesystem. Use this when the command does not rely on the resource being in a filesystem. Region : attempt to infer a Region in a source file. Inference succeeds when the context has a selection of size 1 or larger. For example, when executing a command in an editor that has a selection, the region will be that selection. Offset : attempt to infer an int representing an offset in a source file. Inference succeeds when the context has a cursor offset (i.e., a selection of size 0 or larger). For example, when executing a command in an editor, the offset will be the offset to the cursor in the editor. EnclosingContext($EnclosingCommandContext) attempts to infer the argument by the enclosing context. The following EnclosingCommandContext s are supported: Project : attempt to infer a ResourcePath to the enclosing project. For example, when executing a command in an IDE on a file, directory, or editor for a file, that belongs to a project. Or when executing a command in a CLI, the directory will be the current working directory. Directory : attempt to infer a ResourcePath to the enclosing directory. For example, when executing a command in the context of a file, directory, or editor for a file, the directory will be the parent of that file/directory. Here is an example of a command that shows the parsed AST by taking one file argument that is inferred from context: let showParsedAstCommand = command-def { type = java mb.helloworld.command.HelloWorldShowParsedAstCommand task-def = showParsedAst args-type = java mb.helloworld.task.HelloWorldShowParsedAst.Args display-name = \"Show parsed AST\" description = \"Shows the parsed AST\" supported-execution-types = [Once, Continuous] parameters = [ file = parameter { type = java mb.resource.ResourceKey required = true argument-providers = [Context(ResourceKey)] } ] }","title":"Commands"},{"location":"reference/configuration/#menu-items","text":"Menu items take the form of: a separator representing a horizontal line in a menu used to separate groups of menu items. a menu $Expression [ $MenuItem* ] representing a (nested) menu with a display name defined by the expression which must be a string, and a list of nested menu items. a command-action { $CommandActionOption* } representing an action that executes a command when a user clicks on it. A command action has the following options: Syntax Required? Description Type command-def = $Expression yes The command to execute. Command or qualified Java identifier execution-type = $ExecutionType yes How the command should be executed. n/a required-resource-types = [($ResourceType ,)*] no On which kinds of resources this menu item will be shown on resource context menus. Defaults to empty. If empty, it will not be hidden based on resources. n/a required-enclosing-resource-types = [($EnclosingResourceType ,)*] no On which kinds of enclosing resources this menu item will be shown on resource context menus. Defaults to empty. If empty, it will not be hidden based on enclosing resource. n/a required-editor-file-types = [($EditorFileType ,)*] no On which kinds of editors belonging to certain file types this menu item will be shown. Defaults to empty. If empty, it will not be hidden based on editor file types. n/a required-editor-selection-types = [($EditorSelectionType ,)*] no On which kinds of editor selection types this menu item will be shown. Defaults to empty. If empty, it will not be hidden based on editor selections. n/a display-name = $Expression no The display name of the command action. Defaults to the display name of the command String description = $Expression no The description of he command action. Defaults to the description of the command String The following ResourceType s are supported: Directory : the menu item will only be shown when a directory is selected. File : the menu item will only be shown when a file is selected. The following EnclosingResourceType s are supported: Project : the menu item will only be shown when the selected resource has an enclosing project. Directory: : the menu item will only be shown when the selected resource has an enclosing directory. The following EditorFileType s are supported: HierarchicalResource : the menu item will only be shown when the editor belongs to a hierarchical resource. That is, a resource that belongs to a tree, such as a file or directory on the local filesystem. ReadableResource : the menu item will only be shown when the editor belongs to a readable resource. The following EditorSelectionType s are supported: Region : the menu item will only be shown when a region with size >0 in the source file is selected. Offset : the menu item will only be shown in the context of an editor with a cursor.","title":"Menu items"},{"location":"reference/configuration/#menus","text":"Menu items are assigned to 3 particular menus: editor-context-menu [ $MenuItem* ] : the context menu that gets shown in editors of the language, for example when right-clicking in an editor of the language in an IDE. Spoofax automatically creates a top-level submenu with the name of the language to host the editor context menu items. The required-editor-file-types and required-editor-selection-types options are used to filter menu items. resource-context-menu [ $MenuItem* ] : the context menu that gets shown in resource explorers, for example when right-clicking in the file browser in an IDE. Spoofax automatically creates a top-level submenu with the name of the language to host the editor context menu items. The required-resource-types and required-enclosing-resource-types options are used to filter menu items. main-menu [ $MenuIitem* ] : the main menu of the language, which is shown on the menu bar in IDEs. When no main-menu section is given, it defaults to the same menu as editor-context-menu . For example, we can assign the command defined earlier to several menus: editor-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand execution-type = Once } command-action { command-def = showParsedAstCommand execution-type = Continuous } ] ] resource-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand execution-type = Once required-resource-types = [File] } ] ]","title":"Menus"},{"location":"reference/configuration/#language-feature-sections","text":"","title":"Language feature sections"},{"location":"reference/configuration/#parser","text":"The parser { $ParserOption* } section enables generation of a parser, and groups options. The sdf3 section must be enabled when the parser section is enabled. The following ParserOption s are supported: Syntax Required? Description Type default-start-symbol = $Expression yes The start symbol to use when no specific one is provided. SDF3 sort identifier variant = $ParserVariant no The parser variant to use. Defaults to jsglr1 . n/a The following ParserVariant s are supported: jsglr1 : uses the JSGLR1 parser. jsglr2 { $Jsglr2Option* } : uses the JSGLR2 parser. The following Jsglr2Option s are supported: preset = $Jsglr2Preset : sets the JSGLR2 preset to use. The following Jsglr2Preset s are supported: Standard Elkhound Recovery RecoveryElkhound DataDependent LayoutSensitive Composite Incremental IncrementalRecovery","title":"Parser"},{"location":"reference/configuration/#comment-symbols","text":"The comment-symbols { $CommentSymbolOption* } section enables specification of line and block comment characters, which are required for the \"toggle comment\" editor service. The following CommentSymbolOption s are supported: Syntax Required? Description Type line = $Expression no Adds a line comment symbol. Can be given multiple times to list multiple line comment symbols. The first one will be used to comment a line with the \"toggle comment\" editor service. String block = $Expression * $Expression no Adds block comment symbols, with an opening and close symbol. Current \"toggle comment\" editor services do not use block comment symbols yet. String","title":"Comment symbols"},{"location":"reference/configuration/#bracket-symbols","text":"The bracket-symbols { $BracketSymbolOption* } section enables specification of bracket symbols (e.g., square brackets, curly brackets, parentheses, etc.), which are required for the \"bracket matching\" editor service. The following BracketSymbolOption s are supported: Syntax Required? Description Type bracket = $Expression * $Expression no Adds bracket symbols, with an opening and closing symbol. Can be given multiple times to list multiple bracket symbols. Character","title":"Bracket symbols"},{"location":"reference/configuration/#styler","text":"The styler { $StylerOption* } section enables generation of a styler, and groups options. The esv section must be enabled when the styler section is enabled. Currently, no StylerOption s are supported.","title":"Styler"},{"location":"reference/configuration/#constraint-analyzer","text":"The constraint-analyzer { $ConstraintAnalyzerOption* } section enables generation of a constraint analyzer, and groups options. The statix section must be enabled when the constraint-analyzer section is enabled. The following ConstraintAnalyzerOption s are supported: Syntax Required? Description Type multi-file = $Expression no Whether multi-file analysis is enabled. Defaults to false . Boolean stratego-strategy = $Expression no The stratego strategy entry-point that handles communication with the constraint-solver. Defaults to editor-analyze . Stratego strategy identifier default-statix-message-stacktrace-length = $Expression no The default Statix message stacktrace length to use. Default is implementation-defined. Does nothing if Statix is not enabled. Unsigned integer default-statix-message-term-depth = $Expression no The default Statix message term depth to use. Default is implementation-defined. Does nothing if Statix is not enabled. Unsigned integer default-statix-test-log-level = $Expression no The default Statix test log level to use. Default is implementation-defined. Does nothing if Statix is not enabled. String default-statix-supress-cascading-errors = $Expression no Whether to suppress cascading errors by default. Default is implementation-defined. Does nothing if Statix is not enabled. Boolean statix-solver-mode = $StatixSolverMode no Statix solver mode. Defaults to traditional . Does nothing if Statix is not enabled. n/a The following StatixSolverMode s are supported: traditional concurrent incremental","title":"Constraint analyzer"},{"location":"reference/configuration/#multi-language-analyzer","text":"The multilang-analyzer { $MultilangAnalyzerOption* } section enables generation of a multi-language analyzer, and groups options. The constraint-analyzer and statix sections must be enabled when the multilang-analyzer section is enabled. Currently, no MultilangAnalyzerOption s are supported.","title":"Multi-language analyzer"},{"location":"reference/configuration/#stratego-runtime","text":"The stratego-runtime { $StrategoRuntimeOption* } section enables generation of a stratego runtime, and groups options. The stratego section must be enabled when the stratego-runtime section is enabled. The following StrategoRuntimeOption s are supported: Syntax Required? Description Type strategy-package-id = $Expression no Adds a package as a private Stratego package. Can be specified multiple times. Java package identifier interop-registerer-by-reflection = $Expression no Adds an interop registerer to load by reflection. Can be specified multiple times. Java type identifier add-spoofax2-primitives = $Expression no Whether to add the Spoofax 2 Stratego primitives. Boolean add-nabl2-primitives = $Expression no Whether to add the NaBL2 Stratego primitives. Boolean add-statix-primitives = $Expression no Whether to add the Statix Stratego primitives. Boolean with-primitive-library = $Expression no Adds a Stratego primitive strategies library (implementing org.spoofax.interpreter.library.IOperatorRegistry ) to the generated StrategoRuntimeBuilderFactory . The library must have an @Inject constructor. Can be specified multiple times. Java type identifier with-interop-registerer = $Expression no Adds a Stratego interop registerer (implementing org.strategoxt.lang.InteropRegisterer ) to the generated StrategoRuntimeBuilderFactory . The registerer must have an @Inject constructor. Can be specified multiple times. Java type identifier class-kind = $Expression no Specifies whether the classes are generated ( Generated ) or provided manually ( Manual ). Defaults to Generated . Generated or Manual base-StrategoRuntimeBuilderFactory = $Expression no Package and name of the generated StrategoRuntimeBuilderFactory . Java type identifier extend-StrategoRuntimeBuilderFactory = $Expression no Package and name of the extending StrategoRuntimeBuilderFactory , if any. Java type identifier","title":"Stratego runtime"},{"location":"reference/configuration/#completer","text":"The completer { $CompleterOption* } section enables generation of a code completer, and groups options. The constraint-analyzer and statix sections must be enabled when the completer section is enabled. Currently, no CompleterOption s are supported.","title":"Completer"},{"location":"reference/configuration/#reference-resolution","text":"The reference-resolution { $ReferenceResolutionOption* } section enables generation of the reference resolver editor service, and groups options. The following ReferenceResolutionOption s are supported: Syntax Required? Description Type variant = $ReferenceResolutionVariant yes The reference resolution variant to use. n/a The following ReferenceResolutionVariant s are supported: Stratego-based: stratego { strategy = strategy $Strategy } where Strategy is a Stratego strategy, typically editor-resolve .","title":"Reference resolution"},{"location":"reference/configuration/#hover-tooltips","text":"The hover { $HoverOption* } section enables generation of the hover text editor service, and groups options. The following HoverOption s are supported: Syntax Required? Description Type variant = $HoverVariant yes The reference resolution variant to use. n/a The following HoverVariant s are supported: Stratego-based: stratego { strategy = strategy $Strategy } where Strategy is a Stratego strategy, typically editor-hover .","title":"Hover tooltips"},{"location":"reference/configuration/#meta-language-sections","text":"","title":"Meta-language sections"},{"location":"reference/configuration/#sdf3","text":"The sdf3 { $Sdf3Option* } section enables syntax definition with SDF3 . The parser section must be enabled when the sdf3 section is enabled. The following Sdf3Option s are supported: Syntax Required? Description Type source = $Sdf3Source no The source of the SDF3 definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./start.sdf3 as its main file relative to the main source directory. n/a The following $Sdf3Source s are supported: Files: files { $Sdf3FilesOption* } Prebuilt: prebuilt { $Sdf3PrebuiltOption } The following Sdf3FilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main SDF3 file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main SDF3 file relative to the main-source-directory . Defaults to ./start.sdf3 . Path include-directory = $Expression no Adds an include directory from which to resolve SDF3 imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the SDF3 files in it accessable to dependencies. May be given multiple times. Path parse-table-generator { $ParseTableGeneratorOption* } no Parse table generator options. n/a placeholders { $PlaceholderOption* } no Placeholder syntax used in code completion. n/a stratego-concrete-syntax-extension-main-file = $Expression no Sets the main SDF3 file used to create a concrete syntax extension parse table. Path The following $Sdf3PrebuiltOption s are supported: Syntax Required? Description Type parse-table-aterm-file = $Expression yes The prebuilt SDF3 parse table ATerm file to use (usually called sdf.tbl ) relative to the root directory Path parse-table-persisted-file = $Expression yes The prebuilt SDF3 parse table persisted file to use (usually called sdf.bin ) relative to the root directory Path The following ParseTableGeneratorOption s are supported: Syntax Required? Description Type dynamic = $Expression no Whether the generated parse table is dynamic. Defaults to false . Boolean data-dependent = $Expression no Whether the generated parse table is data-dependent. Defaults to false . Boolean layout-sensitive = $Expression no Whether the generated parse table is layout-sensitive. Defaults to false . Boolean solve-deep-conflicts = $Expression no Whether the parse table generator solves deep priority conflicts. Defaults to true . Boolean check-overlap = $Expression no Whether the parse table generator checks for overlap. Defaults to false . Boolean check-priorities = $Expression no Whether the parse table generator checks priorities. Defaults to false . Boolean The following PlaceholderOption s are supported: Syntax Required? Description Type prefix = $Expression no The prefix to a placholder. Defaults to [[ , or empty if the other is defined. String postfix = $Expression no The postfix to a placeholder. Defaults to ]] , or empty if the other is defined. String","title":"SDF3"},{"location":"reference/configuration/#esv","text":"The esv { $EsvOption* } section enables syntax-based styling definition with ESV . The styler section must be enabled when the esv section is enabled. The following EsvOption s are supported: Syntax Required? Description Type source = $EsvSource no The source of the ESV definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.esv as its main file relative to the main source directory. n/a The following $EsvSource s are supported: Files: files { $EsvFilesOption* } Prebuilt: prebuilt { $EsvPrebuiltOption } The following EsvFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main ESV file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main ESV file relative to the main-source-directory . Defaults to ./main.esv . Path include-directory = $Expression no Adds an include directory from which to resolve ESV imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the ESV files in it accessable to dependencies. May be given multiple times. Path The following $EsvPrebuiltOption s are supported: Syntax Required? Description Type file = $Expression yes The prebuilt ESV file to use relative to the root directory Path","title":"ESV"},{"location":"reference/configuration/#statix","text":"The statix { $StatixOption* } section enables static semantics definition with Statix . The constraint-anaylzer section must be enabled when the statix section is enabled. The following StatixOption s are supported: Syntax Required? Description Type source = $StatixSource no The source of the Statix definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.stx as its main file relative to the main source directory. n/a The following $StatixSource s are supported: Files: files { $StatixFilesOption* } Prebuilt: prebuilt { $StatixPrebuiltOption } The following StatixFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main Statix file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main Statix file relative to the main-source-directory . Defaults to ./main.stx . Path include-directory = $Expression no Adds an include directory from which to resolve Statix imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the Statix files in it accessable to dependencies. May be given multiple times. Path sdf3-statix-signature-generation = $Expression no Whether SDF3 to Statix signature generation is enabled. When enabled, stratego { sdf3-statix-explication-generation = true } must also be enabled. Defaults to false . Boolean The following $StatixPrebuiltOption s are supported: Syntax Required? Description Type spec-aterm-directory = $Expression yes The prebuilt Statix spec ATerm directory to use relative to the root directory Path","title":"Statix"},{"location":"reference/configuration/#stratego","text":"The stratego { $StrategoOption* } section enables definition of transformations with Stratego . The stratego-runtime section must be enabled when the stratego section is enabled. The following StrategoOption s are supported: Syntax Required? Description Type source = $StrategoSource no The source of the Statix definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.str2 as its main file relative to the main source directory. n/a output-java-package = $Expression no The Java package into which compiled Stratego Java files are generated. Defaults to the language's package, followed by .strategies . String The following $StrategoSource s are supported: Files: files { $StrategoFilesOption* } The following $StrategoFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main Stratego file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main Stratego file relative to the main-source-directory . Defaults to ./main.str2 . Path include-directory = $Expression no Adds an include directory from which to resolve Stratego imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the Stratego files in it accessable to dependencies. May be given multiple times. Path with-import-strategy-package-id = $Expression no Adds a Java package to import in the Java files that have been compiled from Stratego files. For example, \"mb.mylang.strategies\" adds import import mb.mylang.strategies.* . May be given multiple times. String sdf3-statix-explication-generation = $Expression no Whether SDF3 to Statix injection explication/implication generation is enabled. When enabled, statix { sdf3-statix-signature-generation = true } must also be enabled. Defaults to false . Boolean language-strategy-affix = $Expression no The affix that is used to make certain generated strategies unique to the language. This is used both as a prefix and suffix. Defaults to name of the language transformed to a Stratego strategy identifier. Stratego strategy identifier concrete-syntax-extension-parse-table = $Expression no Adds a Stratego concrete syntax extension parse table that can be used to parse Stratego files with. May be given multiple times. Path","title":"Stratego"},{"location":"reference/configuration/#dependencies","text":"The dependencies [ $Dependency* ] section allows specifying dependencies to other language (library) projects. A dependency consists of a dependency expression ( $Exp ) that specifies the source of the dependency. The following expressions are supported: $Coordinate : an exact coordinate to a project, such as org.metaborg:strategolib:1.0.0 . Coordinates are resolved to a statically loaded language, or a dynamically loaded language definition. $CoordinateRequirement : a coordinate to a project with an open version, such as org.metaborg:strategolib:* , resolved identically to $Coordinate except that the latest available version will be chosen. $Path : a relative path (relative to the directory spoofaxc.cfg is in) to a language definition. Dependencies can be of a certain kind ( $DependencyKind ): Build : a dependency that is resolved at build time , meaning when the language definition is built, allowing the use of sources and binaries of the project the dependency points to. Run : a dependency that is resolved at run time , meaning when the language is executed, allowing use of the classes and tasks of the project the dependency points to. Note: this kind of dependency has not yet been implemented. The following Dependency s are supported: Syntax Description $Exp A dependency to a project defined by the expression, available at Build and Run time. $Exp { $DependencyOption* } A dependency to project defined by the expression, with configuration options. The following DependencyOption s are supported: kinds = [ $DependencyKind* ] : set the kinds at which this dependency is resolved. Additionally, build time dependencies can be quickly defined with the build-dependencies [ $BuildDependency* ] section. The following BuildDependency s are supported: Syntax Description $Exp A dependency to project defined by the expression, available only at Build time. For example, the following configuration adds build dependencies to common libraries shipped with Spoofax 3: build-dependencies [ org.metaborg : strategolib:* org.metaborg : gpp:* org.metaborg : libspoofax2:* org.metaborg : libstatix:* ]","title":"Dependencies"},{"location":"reference/configuration/#spoofaxclock","text":"The spoofaxc.lock file, which resides next to the spoofaxc.cfg file, contains values for several options that have defaults derived from other options, in order to keep these derived values stable even when the options they are derived from are changed. For example, when no java-class-id-prefix option is set in spoofaxc.cfg , it will be derived from the name option with some changes to make it compatible as a Java identifier, and is stored under shared.defaultClassPrefix in the spoofaxc.lock file. When you change the name of your language, the stored value will be used, keeping the class prefix the same, making it possible to rename the language without having to rename all class files. Therefore, the spoofaxc.lock file should be checked in to source control, in order to have reproducible builds. If you do want to re-derive a default from other options, remove the option from the spoofaxc.lock file and rebuild the language. The value will be re-derived and stored in spoofaxc.lock , after which you need to check it into source control again.","title":"spoofaxc.lock"},{"location":"reference/language_definition/","text":"Language definition \u00b6 In essence, a language definition in Spoofax 3 consists of several source files along with configuration that specify and implement the various aspects of a language, such as its tokenizer, parser, styler, completer, checker, compiler, commands, and so forth. The source files of a language definition are mostly written in high-level domain-specific meta-languages, with some parts being written in Java. The Spoofax 3 compiler compiles language definitions into language implementations which are essentially standard Java libraries (e.g., JAR files) consisting of Java classes and bundled resources. These classes implement the various aspects of a language, and may use bundled resources such as a parse table which is loaded and interpreted at runtime. In this reference manual we explain the basic anatomy of a language definition, its configuration and file structure, Java classes and how they are instantiated, as well as how a language definition is compiled into a language implementation. Anatomy \u00b6 configuration in CFG syntax in SDF3 styling in ESV static semantics in Statix transformations in Stratego tasks in java (future: PIE DSL) commands, specified in configuration, generated for you menu bindings CLI bindings File structure \u00b6 Java classes \u00b6 description of the Java classes, their instances, and how they are instantiated basic classes: ClassLoaderResources ParseTable/ParserFactory/Parser StylingRules/StylerFactory/Styler StrategoRuntimeBuilderFactory ConstraintAnalyzerFactory/ConstraintAnalyzer components: ResourcesComponent/ResourcesModule Component/Module Scope/Qualifier how they are instantiated by Dagger LanguageInstance: tasks command defs auto command request CLI commands menu items task definitions: Tokenize Style Completion Check/CheckMulti/CheckAggregator/CheckDeaggregator Parse Analyze/AnalyzeMulti GetStrategoRuntimeProvider commands: Instantiation \u00b6 dependency injection Compilation \u00b6 description of how a language definition is compiled into a language implementation, and what the compiled form looks like. generate Java sources into: build/generated/sources/language build/generated/sources/adapter generate Stratego sources into: build/generated/sources/languageSpecification/stratego","title":"Language definition"},{"location":"reference/language_definition/#language-definition","text":"In essence, a language definition in Spoofax 3 consists of several source files along with configuration that specify and implement the various aspects of a language, such as its tokenizer, parser, styler, completer, checker, compiler, commands, and so forth. The source files of a language definition are mostly written in high-level domain-specific meta-languages, with some parts being written in Java. The Spoofax 3 compiler compiles language definitions into language implementations which are essentially standard Java libraries (e.g., JAR files) consisting of Java classes and bundled resources. These classes implement the various aspects of a language, and may use bundled resources such as a parse table which is loaded and interpreted at runtime. In this reference manual we explain the basic anatomy of a language definition, its configuration and file structure, Java classes and how they are instantiated, as well as how a language definition is compiled into a language implementation.","title":"Language definition"},{"location":"reference/language_definition/#anatomy","text":"configuration in CFG syntax in SDF3 styling in ESV static semantics in Statix transformations in Stratego tasks in java (future: PIE DSL) commands, specified in configuration, generated for you menu bindings CLI bindings","title":"Anatomy"},{"location":"reference/language_definition/#file-structure","text":"","title":"File structure"},{"location":"reference/language_definition/#java-classes","text":"description of the Java classes, their instances, and how they are instantiated basic classes: ClassLoaderResources ParseTable/ParserFactory/Parser StylingRules/StylerFactory/Styler StrategoRuntimeBuilderFactory ConstraintAnalyzerFactory/ConstraintAnalyzer components: ResourcesComponent/ResourcesModule Component/Module Scope/Qualifier how they are instantiated by Dagger LanguageInstance: tasks command defs auto command request CLI commands menu items task definitions: Tokenize Style Completion Check/CheckMulti/CheckAggregator/CheckDeaggregator Parse Analyze/AnalyzeMulti GetStrategoRuntimeProvider commands:","title":"Java classes"},{"location":"reference/language_definition/#instantiation","text":"dependency injection","title":"Instantiation"},{"location":"reference/language_definition/#compilation","text":"description of how a language definition is compiled into a language implementation, and what the compiled form looks like. generate Java sources into: build/generated/sources/language build/generated/sources/adapter generate Stratego sources into: build/generated/sources/languageSpecification/stratego","title":"Compilation"},{"location":"reference/eclipse-lwb/eclipse-project-files/","text":"Eclipse Project Files \u00b6 To be able to import a Spoofax 3 project into Eclipse, it should have at least the .project and .classpath project files. Those files live in the root of the project (i.e., where the spoofaxc.cfg file lives), with the following minimum content: Eclipse will adjust these files as needed. Ensure these files are not in .gitignore Often these Eclipse files are ignored for version control by specifying them in .gitignore . To allow these projects to be imported in the future, do not ignore them when committing the files. Adjust the myproject to be the name of the project. .project myproject spoofax.lwb.eclipse.builder.project.references spoofax.lwb.eclipse.builder clean,full,incremental, org.eclipse.jdt.core.javabuilder org.eclipse.jdt.core.javanature spoofax.lwb.eclipse.nature .classpath ","title":"Eclipse Project Files"},{"location":"reference/eclipse-lwb/eclipse-project-files/#eclipse-project-files","text":"To be able to import a Spoofax 3 project into Eclipse, it should have at least the .project and .classpath project files. Those files live in the root of the project (i.e., where the spoofaxc.cfg file lives), with the following minimum content: Eclipse will adjust these files as needed. Ensure these files are not in .gitignore Often these Eclipse files are ignored for version control by specifying them in .gitignore . To allow these projects to be imported in the future, do not ignore them when committing the files. Adjust the myproject to be the name of the project. .project myproject spoofax.lwb.eclipse.builder.project.references spoofax.lwb.eclipse.builder clean,full,incremental, org.eclipse.jdt.core.javabuilder org.eclipse.jdt.core.javanature spoofax.lwb.eclipse.nature .classpath ","title":"Eclipse Project Files"},{"location":"release/download/","text":"Downloads \u00b6 This page contains download links to the latest release and development version of Spoofax 3. Latest release \u00b6 The latest release of Spoofax is 0.22.0, released on 27-05-2024. We recommend the use of releases because we make sure to make them available indefinitely. Follow the Installation tutorial for instructions on how to install and run Spoofax. Eclipse Language Workbench Environment \u00b6 With embedded JVM: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM Without embedded JVM: Windows 64-bit macOS 64-bit Linux 64-bit Repository for installing into an existing Eclipse installation: https://artifacts.metaborg.org/content/unzip/releases-unzipped/org/metaborg/spoofax.lwb.eclipse.repository/0.22.0/spoofax.lwb.eclipse.repository-0.22.0.zip-unzip/ Development version \u00b6 The development version of Spoofax is always develop-SNAPSHOT . Development versions of Spoofax use snapshot versioning, meaning that they always point to the latest build that is made whenever a change is pushed to Spoofax\u2019s repositories. Therefore, older builds are not available since they are continuously replaced by newer builds. We do not recommend usage of the development version unless you absolutely need its features, want to help us test, or if you know what you are doing. Eclipse Language Workbench Environment \u00b6 With embedded JVM: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM Without embedded JVM: Windows 64-bit macOS 64-bit Linux 64-bit","title":"Downloads"},{"location":"release/download/#downloads","text":"This page contains download links to the latest release and development version of Spoofax 3.","title":"Downloads"},{"location":"release/download/#latest-release","text":"The latest release of Spoofax is 0.22.0, released on 27-05-2024. We recommend the use of releases because we make sure to make them available indefinitely. Follow the Installation tutorial for instructions on how to install and run Spoofax.","title":"Latest release"},{"location":"release/download/#eclipse-language-workbench-environment","text":"With embedded JVM: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM Without embedded JVM: Windows 64-bit macOS 64-bit Linux 64-bit Repository for installing into an existing Eclipse installation: https://artifacts.metaborg.org/content/unzip/releases-unzipped/org/metaborg/spoofax.lwb.eclipse.repository/0.22.0/spoofax.lwb.eclipse.repository-0.22.0.zip-unzip/","title":"Eclipse Language Workbench Environment"},{"location":"release/download/#development-version","text":"The development version of Spoofax is always develop-SNAPSHOT . Development versions of Spoofax use snapshot versioning, meaning that they always point to the latest build that is made whenever a change is pushed to Spoofax\u2019s repositories. Therefore, older builds are not available since they are continuously replaced by newer builds. We do not recommend usage of the development version unless you absolutely need its features, want to help us test, or if you know what you are doing.","title":"Development version"},{"location":"release/download/#eclipse-language-workbench-environment_1","text":"With embedded JVM: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM Without embedded JVM: Windows 64-bit macOS 64-bit Linux 64-bit","title":"Eclipse Language Workbench Environment"},{"location":"tutorial/add_transformation/","text":"Adding a transformation \u00b6 Finally, we will define a transformation for our language and add a task, command-tasks, command, and menu item for it. Open the main Stratego file helloworld/src/main.str . Stratego . is a meta-language for defining term (AST) transformations through rewrite rules. We will add a silly transformation that replaces all instances of World () with Hello () . Add the following code to the end of the Stratego file: rules replace-world : Hello () - > Hello () replace-world : World () - > Hello () replace-worlds = topdown ( try ( replace-world )) The replace-world rule passes Hello () terms but rewrites World () terms to Hello () . The replace-worlds strategy tries to apply replace-world in a top-down manner over the entire AST. src/main.str full contents module main imports statixruntime statix / api injections / - signatures / - rules // Analysis pre-analyze = explicate-injections-helloworld-Start post-analyze = implicate-injections-helloworld-Start editor-analyze = stx-editor-analyze ( pre-analyze , post-analyze | \"main\" , \"programOk\" ) rules replace-world : Hello () - > Hello () replace-world : World () - > Hello () replace-worlds = topdown ( try ( replace-world )) ``` Now we add a task and command-task for this transformation . We define two separate tasks to keep separate 1 . the act of transforming the program , and 2 . feeding back the result of that transformation to the user that executes a command . This practice later allows us to reuse the first task in a different task if we need to . Right-click the ` mb . helloworld . task ` package and create the ` HelloWorldReplaceWorlds ` class and replace the entire Java file with : ``` { . java . annotate linenums = \"1\" } package mb . helloworld . task ; import mb . helloworld . HelloWorldClassLoaderResources ; import mb . helloworld . HelloWorldScope ; import mb . pie . api . ExecContext ; import mb . pie . api . stamp . resource . ResourceStampers ; import mb . stratego . pie . AstStrategoTransformTaskDef ; import javax . inject . Inject ; import java . io . IOException ; @ HelloWorldScope public class HelloWorldReplaceWorlds extends AstStrategoTransformTaskDef { private final HelloWorldClassLoaderResources classloaderResources ; @ Inject public HelloWorldReplaceWorlds ( // 1 HelloWorldClassLoaderResources classloaderResources , HelloWorldGetStrategoRuntimeProvider getStrategoRuntimeProvider ) { super ( getStrategoRuntimeProvider , \"replace-worlds\" ); // 2 this . classloaderResources = classloaderResources ; } @ Override public String getId () { // 3 return getClass () . getName (); } @ Override protected void createDependencies ( ExecContext context ) throws IOException { // 4 context . require ( classloaderResources . tryGetAsLocalResource ( getClass ()), ResourceStampers . hashFile ()); } } This task extends AstStrategoTransformTaskDef which is a convenient abstract class for creating tasks that run Stratego transformations by implementing a constructor and a couple of methods: The constructor should inject HelloWorldClassLoaderResources which we again will use to create a self-dependency, and HelloWorldGetStrategoRuntimeProvider which is a task that Spoofax generates for your language, which provides a Stratego runtime to execute strategies with. The HelloWorldGetStrategoRuntimeProvider instance is provided to the superclass constructor, along with the strategy that we want this task to execute, which is \"replace-worlds\" . We override getId of TaskDef again to give this task a unique identifier. We override createDependencies of AstStrategoTransformTaskDef to create a self-dependency. Then create the HelloWorldShowReplaceWorlds class and replace the entire Java file with: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 package mb.helloworld.task ; import java.io.Serializable ; import java.util.Objects ; import javax.inject.Inject ; import org.checkerframework.checker.nullness.qual.Nullable ; import mb.helloworld.HelloWorldClassLoaderResources ; import mb.helloworld.HelloWorldScope ; import mb.pie.api.ExecContext ; import mb.pie.api.TaskDef ; import mb.pie.api.stamp.resource.ResourceStampers ; import mb.resource.ResourceKey ; import mb.spoofax.core.language.command.CommandFeedback ; import mb.spoofax.core.language.command.ShowFeedback ; import mb.aterm.common.TermToString ; @HelloWorldScope public class HelloWorldShowReplaceWorlds implements TaskDef < HelloWorldShowReplaceWorlds . Args , CommandFeedback > { public static class Args implements Serializable { private static final long serialVersionUID = 1L ; public final ResourceKey file ; public Args ( ResourceKey file ) { this . file = file ; } @Override public boolean equals ( @Nullable Object o ) { if ( this == o ) return true ; if ( o == null || getClass () != o . getClass ()) return false ; final Args args = ( Args ) o ; return file . equals ( args . file ); } @Override public int hashCode () { return Objects . hash ( file ); } @Override public String toString () { return \"Args{\" + \"file=\" + file + '}' ; } } private final HelloWorldClassLoaderResources classloaderResources ; private final HelloWorldParse parse ; private final HelloWorldReplaceWorlds replaceWorlds ; @Inject public HelloWorldShowReplaceWorlds ( HelloWorldClassLoaderResources classloaderResources , HelloWorldParse parse , HelloWorldReplaceWorlds replaceWorlds ) { this . classloaderResources = classloaderResources ; this . parse = parse ; this . replaceWorlds = replaceWorlds ; } @Override public CommandFeedback exec ( ExecContext context , Args args ) throws Exception { context . require ( classloaderResources . tryGetAsLocalResource ( getClass ()), ResourceStampers . hashFile ()); final ResourceKey file = args . file ; return context . require ( replaceWorlds , parse . inputBuilder (). withFile ( file ). buildAstSupplier ()). mapOrElse ( ast -> CommandFeedback . of ( ShowFeedback . showText ( TermToString . toString ( ast ), \"Replaced World()s with Hello()s for '\" + file + \"'\" )), e -> CommandFeedback . ofTryExtractMessagesFrom ( e , file ) ); } @Override public String getId () { return getClass (). getName (); } } This class very similar to HelloWorldShowParsedAst , but runs the HelloWorldReplaceWorlds task on the parsed AST, transforming the AST. Now open helloworld/spoofax.cfg again and register the tasks by adding: task-def mb.helloworld.task.HelloWorldReplaceWorlds let showReplaceWorlds = task-def mb.helloworld.task.HelloWorldShowReplaceWorlds Then add a command for it by adding: let showReplaceWorldsCommand = command-def { task-def = showReplaceWorlds display-name = \"Replace world with hello\" parameters = [ file = parameter { type = java mb.resource.ResourceKey argument-providers = [Context(ReadableResource)] } ] } Finally, add menu items for the command by adding: editor-context-menu [ menu \"Transform\" [ command-action { command-def = showReplaceWorldsCommand execution-type = Once } command-action { command-def = showReplaceWorldsCommand execution-type = Continuous } ] ] resource-context-menu [ menu \"Transform\" [ command-action { command-def = showReplaceWorldsCommand execution-type = Once required-resource-types = [File] } ] ] Build the project so that we can test our changes. Test the command similarly to testing the \"Show parsed AST\" command.","title":"Adding a transformation"},{"location":"tutorial/add_transformation/#adding-a-transformation","text":"Finally, we will define a transformation for our language and add a task, command-tasks, command, and menu item for it. Open the main Stratego file helloworld/src/main.str . Stratego . is a meta-language for defining term (AST) transformations through rewrite rules. We will add a silly transformation that replaces all instances of World () with Hello () . Add the following code to the end of the Stratego file: rules replace-world : Hello () - > Hello () replace-world : World () - > Hello () replace-worlds = topdown ( try ( replace-world )) The replace-world rule passes Hello () terms but rewrites World () terms to Hello () . The replace-worlds strategy tries to apply replace-world in a top-down manner over the entire AST. src/main.str full contents module main imports statixruntime statix / api injections / - signatures / - rules // Analysis pre-analyze = explicate-injections-helloworld-Start post-analyze = implicate-injections-helloworld-Start editor-analyze = stx-editor-analyze ( pre-analyze , post-analyze | \"main\" , \"programOk\" ) rules replace-world : Hello () - > Hello () replace-world : World () - > Hello () replace-worlds = topdown ( try ( replace-world )) ``` Now we add a task and command-task for this transformation . We define two separate tasks to keep separate 1 . the act of transforming the program , and 2 . feeding back the result of that transformation to the user that executes a command . This practice later allows us to reuse the first task in a different task if we need to . Right-click the ` mb . helloworld . task ` package and create the ` HelloWorldReplaceWorlds ` class and replace the entire Java file with : ``` { . java . annotate linenums = \"1\" } package mb . helloworld . task ; import mb . helloworld . HelloWorldClassLoaderResources ; import mb . helloworld . HelloWorldScope ; import mb . pie . api . ExecContext ; import mb . pie . api . stamp . resource . ResourceStampers ; import mb . stratego . pie . AstStrategoTransformTaskDef ; import javax . inject . Inject ; import java . io . IOException ; @ HelloWorldScope public class HelloWorldReplaceWorlds extends AstStrategoTransformTaskDef { private final HelloWorldClassLoaderResources classloaderResources ; @ Inject public HelloWorldReplaceWorlds ( // 1 HelloWorldClassLoaderResources classloaderResources , HelloWorldGetStrategoRuntimeProvider getStrategoRuntimeProvider ) { super ( getStrategoRuntimeProvider , \"replace-worlds\" ); // 2 this . classloaderResources = classloaderResources ; } @ Override public String getId () { // 3 return getClass () . getName (); } @ Override protected void createDependencies ( ExecContext context ) throws IOException { // 4 context . require ( classloaderResources . tryGetAsLocalResource ( getClass ()), ResourceStampers . hashFile ()); } } This task extends AstStrategoTransformTaskDef which is a convenient abstract class for creating tasks that run Stratego transformations by implementing a constructor and a couple of methods: The constructor should inject HelloWorldClassLoaderResources which we again will use to create a self-dependency, and HelloWorldGetStrategoRuntimeProvider which is a task that Spoofax generates for your language, which provides a Stratego runtime to execute strategies with. The HelloWorldGetStrategoRuntimeProvider instance is provided to the superclass constructor, along with the strategy that we want this task to execute, which is \"replace-worlds\" . We override getId of TaskDef again to give this task a unique identifier. We override createDependencies of AstStrategoTransformTaskDef to create a self-dependency. Then create the HelloWorldShowReplaceWorlds class and replace the entire Java file with: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 package mb.helloworld.task ; import java.io.Serializable ; import java.util.Objects ; import javax.inject.Inject ; import org.checkerframework.checker.nullness.qual.Nullable ; import mb.helloworld.HelloWorldClassLoaderResources ; import mb.helloworld.HelloWorldScope ; import mb.pie.api.ExecContext ; import mb.pie.api.TaskDef ; import mb.pie.api.stamp.resource.ResourceStampers ; import mb.resource.ResourceKey ; import mb.spoofax.core.language.command.CommandFeedback ; import mb.spoofax.core.language.command.ShowFeedback ; import mb.aterm.common.TermToString ; @HelloWorldScope public class HelloWorldShowReplaceWorlds implements TaskDef < HelloWorldShowReplaceWorlds . Args , CommandFeedback > { public static class Args implements Serializable { private static final long serialVersionUID = 1L ; public final ResourceKey file ; public Args ( ResourceKey file ) { this . file = file ; } @Override public boolean equals ( @Nullable Object o ) { if ( this == o ) return true ; if ( o == null || getClass () != o . getClass ()) return false ; final Args args = ( Args ) o ; return file . equals ( args . file ); } @Override public int hashCode () { return Objects . hash ( file ); } @Override public String toString () { return \"Args{\" + \"file=\" + file + '}' ; } } private final HelloWorldClassLoaderResources classloaderResources ; private final HelloWorldParse parse ; private final HelloWorldReplaceWorlds replaceWorlds ; @Inject public HelloWorldShowReplaceWorlds ( HelloWorldClassLoaderResources classloaderResources , HelloWorldParse parse , HelloWorldReplaceWorlds replaceWorlds ) { this . classloaderResources = classloaderResources ; this . parse = parse ; this . replaceWorlds = replaceWorlds ; } @Override public CommandFeedback exec ( ExecContext context , Args args ) throws Exception { context . require ( classloaderResources . tryGetAsLocalResource ( getClass ()), ResourceStampers . hashFile ()); final ResourceKey file = args . file ; return context . require ( replaceWorlds , parse . inputBuilder (). withFile ( file ). buildAstSupplier ()). mapOrElse ( ast -> CommandFeedback . of ( ShowFeedback . showText ( TermToString . toString ( ast ), \"Replaced World()s with Hello()s for '\" + file + \"'\" )), e -> CommandFeedback . ofTryExtractMessagesFrom ( e , file ) ); } @Override public String getId () { return getClass (). getName (); } } This class very similar to HelloWorldShowParsedAst , but runs the HelloWorldReplaceWorlds task on the parsed AST, transforming the AST. Now open helloworld/spoofax.cfg again and register the tasks by adding: task-def mb.helloworld.task.HelloWorldReplaceWorlds let showReplaceWorlds = task-def mb.helloworld.task.HelloWorldShowReplaceWorlds Then add a command for it by adding: let showReplaceWorldsCommand = command-def { task-def = showReplaceWorlds display-name = \"Replace world with hello\" parameters = [ file = parameter { type = java mb.resource.ResourceKey argument-providers = [Context(ReadableResource)] } ] } Finally, add menu items for the command by adding: editor-context-menu [ menu \"Transform\" [ command-action { command-def = showReplaceWorldsCommand execution-type = Once } command-action { command-def = showReplaceWorldsCommand execution-type = Continuous } ] ] resource-context-menu [ menu \"Transform\" [ command-action { command-def = showReplaceWorldsCommand execution-type = Once required-resource-types = [File] } ] ] Build the project so that we can test our changes. Test the command similarly to testing the \"Show parsed AST\" command.","title":"Adding a transformation"},{"location":"tutorial/change_static_semantics/","text":"Changing the static semantics \u00b6 Now we will change the static semantics of the language. Open the main Statix file helloworld/src/main.stx Statix is a meta-language for defining the static semantics of your language, which includes type checking. First we will update the Statix specification to handle the new language constructs. Replace the programOk ( _ ). line with programOk(Program(parts)) :- partsOk(parts). , meaning that we accept programs consisting of parts, as long as their parts are ok. As a silly rule, we will add a warning to all instances of world in the program. Add the following code to the end of the Statix definition: partOk : Part partOk ( Hello ()). partOk ( World ()) : - try { false } | warning $[ World !] . partsOk maps partOk ( list ( * )) This adds a partOk rule that lets all Hello () parts pass, but will add a warning to all World () parts. partsOk goes over a list of parts and applies partOk . src/main.stx full contents module main imports signatures / start-sig rules programOk : Start programOk ( Program ( parts )) : - partsOk ( parts ). partOk : Part partOk ( Hello ()). partOk ( World ()) : - try { false } | warning $[ World !] . partsOk maps partOk ( list ( * )) Build the project, and a warning marker should appear under all instances of world in the example program.","title":"Change static semantics"},{"location":"tutorial/change_static_semantics/#changing-the-static-semantics","text":"Now we will change the static semantics of the language. Open the main Statix file helloworld/src/main.stx Statix is a meta-language for defining the static semantics of your language, which includes type checking. First we will update the Statix specification to handle the new language constructs. Replace the programOk ( _ ). line with programOk(Program(parts)) :- partsOk(parts). , meaning that we accept programs consisting of parts, as long as their parts are ok. As a silly rule, we will add a warning to all instances of world in the program. Add the following code to the end of the Statix definition: partOk : Part partOk ( Hello ()). partOk ( World ()) : - try { false } | warning $[ World !] . partsOk maps partOk ( list ( * )) This adds a partOk rule that lets all Hello () parts pass, but will add a warning to all World () parts. partsOk goes over a list of parts and applies partOk . src/main.stx full contents module main imports signatures / start-sig rules programOk : Start programOk ( Program ( parts )) : - partsOk ( parts ). partOk : Part partOk ( Hello ()). partOk ( World ()) : - try { false } | warning $[ World !] . partsOk maps partOk ( list ( * )) Build the project, and a warning marker should appear under all instances of world in the example program.","title":"Changing the static semantics"},{"location":"tutorial/create_language_project/","text":"Creating a language project \u00b6 This tutorial gets you started with language development by creating a language project and changing various aspects of the language. First follow the installation tutorial if you haven't done so yet. Creating a new project \u00b6 In Eclipse, open the new project dialog by choosing File \u2023 New \u2023 Project from the main menu. In the new project dialog, select Spoofax LWB \u2023 Spoofax language project and press Next . In this wizard, you can customize the various names your language will use. However, for the purpose of this tutorial, fill in HelloWorld as the name of the project, which will automatically fill in the other elements with defaults. Then press Finish to create the project. There should now be a project named helloworld in the Package Explorer . Adding syntax \u00b6 First we will add some syntax to the language. Open the main SDF3 file helloworld/src/start.sdf3 file by expanding the folders and double-clicking the file. SDF3 is a meta-language (i.e., a language to describe languages) for describing the syntax of a language, from which Spoofax will derive the parser of your language. Under the context-free syntax section, replace the Start . Empty = <> line with Start . Program = <<{ Part \" \" } * >> , indicating that the language accepts programs which consists of zero or more parts. Part is a sort and must be defined by adding its name to the context-free sorts section on a new line. Now we will add syntax productions to Part to the context-free syntax section. Add Part . Hello = < hello > on a new line, indicating that one sort of Part is the word hello. Then add Part . World = < world > on a new line, indicating that one sort of Part is the word world . src/start.sdf3 full contents module start context-free start-symbols Start context-free sorts Start Part context-free syntax Start . Program = <<{ Part \" \" } * >> Part . Hello = < hello > Part . World = < world > lexical syntax LAYOUT = [ \\ \\ n \\ v \\ f \\ r ] context-free restrictions LAYOUT ? - / - [ \\ \\ n \\ v \\ f \\ r ] To observe our changes, build the project by clicking on the project in the Package Explorer and choosing Project \u2023 Build Project from the main menu, or by pressing Cmd + B on macOS or Ctrl + B on others. To see when the build is done, open the progress window by choosing Window \u2023 Show View \u2023 Progress . If the progress view is empty, the build is done. The initial build can be a bit slow because there is a lot of code to compile in the background. Subsequent builds will be faster due to incrementalization. Create an example file for your language by right-clicking the project and choosing New \u2023 File , filling in example/example1.hel as file name, and pressing Finish . Type a valid sentence such as hello world hello hello world in this file, and it will highlight purple indicating that hello and world are keywords. We can also run a debugging command on the example file to check the AST that the parser produces. There are three ways to run this debugging command: Make sure the example1.hel editor is open and active (i.e., has focus), and choose from the main menu: Spoofax \u2023 Debug \u2023 Show parsed AST . Make sure the example1.hel editor is open and active (i.e., has focus), right-click in the editor and choose: HelloWorld \u2023 Debug \u2023 Show parsed AST . Right-click the example/example1.hel file in the Package Explorer and choose: HelloWorld \u2023 Debug \u2023 Show parsed AST Running this command will open a new editor with the AST of the program, such as: Program ([ Hello (), World ()]) There are also (continuous) variants of these debugging commands which will keep updating the debugging view when the source program changes. You can drag tabs to the sides of the screen to keep multiple editors open simultaneously, for example to keep the continuous debugging view visible, and to keep the syntax definition files editable. Warning There is currently a bug where continuous debugging views are not updated any more after the language is rebuilt. In that case, you have to open the continuous debugging view again. Testing the new syntax \u00b6 We can also systematically test the syntax (and other facets) of the language with the Spoofax Testing Language (SPT) . Open the SPT file helloworld/test/test.spt . This file contains one test which tests that the empty program parses successfully, which is still the case because a program can consist of 0 parts. Add a new test case to the test suite by adding: test hello world parse [[ hello world ]] parse succeeds which tests that hello world parses successfully. You can also add negative tests such as: test gibberish [[ asdfasdfasdf ]] parse fails test/test.spt full contents module test test parse empty [[]] parse succeeds test hello world parse [[ hello world ]] parse succeeds test gibberish [[ asdfasdfasdf ]] parse fails If you keep the SPT file open and rebuild your language, the SPT tests will be re-executed to provide feedback whether your change to the language conforms to your tests. You can also run all SPT tests by right-clicking a directory with SPT files, or by right-clicking the language project, and choosing SPT \u2023 Run SPT tests . This will (once the tests are done executing) pop up a SPT Test Runner view with the results of testing. Changing syntax highlighting \u00b6 Now we will change the syntax highlighter of the language. Open the main ESV file helloworld/src/main.esv . ESV is a meta-language for describing the syntax highlighter. Change the keyword : 127 0 85 bold line to keyword : 0 0 150 bold and build the project. Then check your example1.hel example file, it should now be highlighted blue. To make iteration easier, you can drag the example1.hel tab to the side of the screen to open the language definition and example file side-by-side. You can play around with the coloring a bit and choose a style to your liking. Remember to rebuild the project after making a change to the language definition. Troubleshooting \u00b6 Check the troubleshooting guide if you run into problems.","title":"Creating a language project"},{"location":"tutorial/create_language_project/#creating-a-language-project","text":"This tutorial gets you started with language development by creating a language project and changing various aspects of the language. First follow the installation tutorial if you haven't done so yet.","title":"Creating a language project"},{"location":"tutorial/create_language_project/#creating-a-new-project","text":"In Eclipse, open the new project dialog by choosing File \u2023 New \u2023 Project from the main menu. In the new project dialog, select Spoofax LWB \u2023 Spoofax language project and press Next . In this wizard, you can customize the various names your language will use. However, for the purpose of this tutorial, fill in HelloWorld as the name of the project, which will automatically fill in the other elements with defaults. Then press Finish to create the project. There should now be a project named helloworld in the Package Explorer .","title":"Creating a new project"},{"location":"tutorial/create_language_project/#adding-syntax","text":"First we will add some syntax to the language. Open the main SDF3 file helloworld/src/start.sdf3 file by expanding the folders and double-clicking the file. SDF3 is a meta-language (i.e., a language to describe languages) for describing the syntax of a language, from which Spoofax will derive the parser of your language. Under the context-free syntax section, replace the Start . Empty = <> line with Start . Program = <<{ Part \" \" } * >> , indicating that the language accepts programs which consists of zero or more parts. Part is a sort and must be defined by adding its name to the context-free sorts section on a new line. Now we will add syntax productions to Part to the context-free syntax section. Add Part . Hello = < hello > on a new line, indicating that one sort of Part is the word hello. Then add Part . World = < world > on a new line, indicating that one sort of Part is the word world . src/start.sdf3 full contents module start context-free start-symbols Start context-free sorts Start Part context-free syntax Start . Program = <<{ Part \" \" } * >> Part . Hello = < hello > Part . World = < world > lexical syntax LAYOUT = [ \\ \\ n \\ v \\ f \\ r ] context-free restrictions LAYOUT ? - / - [ \\ \\ n \\ v \\ f \\ r ] To observe our changes, build the project by clicking on the project in the Package Explorer and choosing Project \u2023 Build Project from the main menu, or by pressing Cmd + B on macOS or Ctrl + B on others. To see when the build is done, open the progress window by choosing Window \u2023 Show View \u2023 Progress . If the progress view is empty, the build is done. The initial build can be a bit slow because there is a lot of code to compile in the background. Subsequent builds will be faster due to incrementalization. Create an example file for your language by right-clicking the project and choosing New \u2023 File , filling in example/example1.hel as file name, and pressing Finish . Type a valid sentence such as hello world hello hello world in this file, and it will highlight purple indicating that hello and world are keywords. We can also run a debugging command on the example file to check the AST that the parser produces. There are three ways to run this debugging command: Make sure the example1.hel editor is open and active (i.e., has focus), and choose from the main menu: Spoofax \u2023 Debug \u2023 Show parsed AST . Make sure the example1.hel editor is open and active (i.e., has focus), right-click in the editor and choose: HelloWorld \u2023 Debug \u2023 Show parsed AST . Right-click the example/example1.hel file in the Package Explorer and choose: HelloWorld \u2023 Debug \u2023 Show parsed AST Running this command will open a new editor with the AST of the program, such as: Program ([ Hello (), World ()]) There are also (continuous) variants of these debugging commands which will keep updating the debugging view when the source program changes. You can drag tabs to the sides of the screen to keep multiple editors open simultaneously, for example to keep the continuous debugging view visible, and to keep the syntax definition files editable. Warning There is currently a bug where continuous debugging views are not updated any more after the language is rebuilt. In that case, you have to open the continuous debugging view again.","title":"Adding syntax"},{"location":"tutorial/create_language_project/#testing-the-new-syntax","text":"We can also systematically test the syntax (and other facets) of the language with the Spoofax Testing Language (SPT) . Open the SPT file helloworld/test/test.spt . This file contains one test which tests that the empty program parses successfully, which is still the case because a program can consist of 0 parts. Add a new test case to the test suite by adding: test hello world parse [[ hello world ]] parse succeeds which tests that hello world parses successfully. You can also add negative tests such as: test gibberish [[ asdfasdfasdf ]] parse fails test/test.spt full contents module test test parse empty [[]] parse succeeds test hello world parse [[ hello world ]] parse succeeds test gibberish [[ asdfasdfasdf ]] parse fails If you keep the SPT file open and rebuild your language, the SPT tests will be re-executed to provide feedback whether your change to the language conforms to your tests. You can also run all SPT tests by right-clicking a directory with SPT files, or by right-clicking the language project, and choosing SPT \u2023 Run SPT tests . This will (once the tests are done executing) pop up a SPT Test Runner view with the results of testing.","title":"Testing the new syntax"},{"location":"tutorial/create_language_project/#changing-syntax-highlighting","text":"Now we will change the syntax highlighter of the language. Open the main ESV file helloworld/src/main.esv . ESV is a meta-language for describing the syntax highlighter. Change the keyword : 127 0 85 bold line to keyword : 0 0 150 bold and build the project. Then check your example1.hel example file, it should now be highlighted blue. To make iteration easier, you can drag the example1.hel tab to the side of the screen to open the language definition and example file side-by-side. You can play around with the coloring a bit and choose a style to your liking. Remember to rebuild the project after making a change to the language definition.","title":"Changing syntax highlighting"},{"location":"tutorial/create_language_project/#troubleshooting","text":"Check the troubleshooting guide if you run into problems.","title":"Troubleshooting"},{"location":"tutorial/install/","text":"Installing the Spoofax 3 language workbench \u00b6 This tutorial gets you set up for language development in Spoofax 3 by installing the Spoofax 3 Eclipse LWB environment. Requirements \u00b6 Spoofax 3 runs on the major operating systems: Windows (64 bits) macOS (64 bits) Linux (64 bits) Other than that, nothing is required as everything is contained in the archive we are going to download. Download \u00b6 To get started, we will download a premade Eclipse installation that comes bundled with the Spoofax 3 LWB plugin. We will download version 0.22.0 released on 27-05-2024. Download the latest version for your platform: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM These are bundled with an embedded JVM so that you do not need to have a JVM installed. If your system has a JVM of version 17 (exactly, higher versions are not supported), and would rather use that, use the following download links instead: Windows 64-bit macOS 64-bit Linux 64-bit Unpack \u00b6 Unpack the downloaded archive to a directory with write access . Write access is required because Eclipse needs to write to several configuration files inside its installation. Warning The unpacked directory or application may be renamed, but do not include spaces or other characters that would not be allowed in a URI (i.e., : ? # [ ] @ ). The same is true for the directory the archive is extracted to. Failing to do so breaks a built-in classpath detection mechanism which will cause Java compilation errors. Warning On Windows do not unpack the Eclipse installation into Program Files , because no write access is granted there, breaking both Eclipse and Spoofax. On macOS Sierra (10.12) and above, after unpacking, open the Terminal and navigate to the directory where the Spoofax3.app file is located and execute: xattr -rc Spoofax3.app Running Eclipse \u00b6 Start up Eclipse, depending on your operating system: Windows: run Spoofax3/eclipse.exe macOS run Spoofax3.app Linux run Spoofax3/eclipse Warning macOS Sierra (10.12) and above will mark the unpacked Spoofax3.app as \"damaged\" due to a modified signed/notarized application, because we have modified the eclipse.ini file inside it. To fix this, open the Terminal, navigate to the directory where the Spoofax3.app file is located, and execute: xattr -rc Spoofax3.app After starting up, choose where your workspace will be stored. The Eclipse workspace will contain all of your settings, and is the default location for new projects. Warning Currently, there are several bugs regarding spaces in the workspace path, so ensure there are no spaces on the workspace path. Configuring Eclipse's preferences \u00b6 Some Eclipse preferences unfortunately have sub-optimal defaults. After you have chosen a workspace and Eclipse has completely started up (and you have closed the Welcome page), go to the Eclipse preferences by pressing Cmd + , on macOS and by going to the Window \u2023 Preferences menu on others, and set these options: General \u2023 Startup and Shutdown Enable: Refresh workspace on startup General \u2023 Workspace Enable: Refresh using native hooks or polling General \u2023 Workspace \u2023 Build Enable: Save automatically before manual build We need to make sure that Eclipse has detected an installed JRE . Open the Eclipse preferences and go to the Java \u2023 Installed JREs page: If there are no installed JREs, and you've downloaded an Eclipse installation with an embedded JVM , press Search... and navigate to the location where you unpacked the Eclipse installation, and choose the jvm directory in it. Then press the checkmark of the JRE to activate it. If there are no installed JREs, and you've downloaded an Eclipse installation without an embedded JVM , press Search... and navigate to the location where your JVM installed, and choose it. Then press the checkmark of the JRE to activate it. If there are one or more installed JVMs, but none are selected, select an appropriate one by pressing the checkmark. If there are one or more installed JVMs, and one is selected, you are good to go. Finally, you may configure Eclipse to your liking. Some typical settings to adjust: General \u2023 Editors \u2023 Text Editors Displayed tab width : change to your desired tab width. Most of Spoofax uses 2 by convention. Insert spaces for tabs : enable, as Spoofax uses spaces by convention. Show print margin : enable and set if you want to set a maximum line length. Show line numbers : enable if you want to see line numbers. General \u2023 Appearance : choose a Theme to your liking. General \u2023 Appearance \u2023 Colors and Fons \u2023 Text Font : the JetBrains Mono font supports ligatures and is used in (most of) the lecture slides (install separately). General \u2023 Keys : change keybindings. Tip These preferences are stored per workspace. If you create a fresh workspace, you have to re-do these settings. You can create a new workspace with copied preferences by selecting File \u2023 Switch workspace \u2023 Other... , and then checking Preferences under Copy settings . Now that Eclipse is set up, continue with creating a language project","title":"Installing the Spoofax 3 language workbench"},{"location":"tutorial/install/#installing-the-spoofax-3-language-workbench","text":"This tutorial gets you set up for language development in Spoofax 3 by installing the Spoofax 3 Eclipse LWB environment.","title":"Installing the Spoofax 3 language workbench"},{"location":"tutorial/install/#requirements","text":"Spoofax 3 runs on the major operating systems: Windows (64 bits) macOS (64 bits) Linux (64 bits) Other than that, nothing is required as everything is contained in the archive we are going to download.","title":"Requirements"},{"location":"tutorial/install/#download","text":"To get started, we will download a premade Eclipse installation that comes bundled with the Spoofax 3 LWB plugin. We will download version 0.22.0 released on 27-05-2024. Download the latest version for your platform: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM These are bundled with an embedded JVM so that you do not need to have a JVM installed. If your system has a JVM of version 17 (exactly, higher versions are not supported), and would rather use that, use the following download links instead: Windows 64-bit macOS 64-bit Linux 64-bit","title":"Download"},{"location":"tutorial/install/#unpack","text":"Unpack the downloaded archive to a directory with write access . Write access is required because Eclipse needs to write to several configuration files inside its installation. Warning The unpacked directory or application may be renamed, but do not include spaces or other characters that would not be allowed in a URI (i.e., : ? # [ ] @ ). The same is true for the directory the archive is extracted to. Failing to do so breaks a built-in classpath detection mechanism which will cause Java compilation errors. Warning On Windows do not unpack the Eclipse installation into Program Files , because no write access is granted there, breaking both Eclipse and Spoofax. On macOS Sierra (10.12) and above, after unpacking, open the Terminal and navigate to the directory where the Spoofax3.app file is located and execute: xattr -rc Spoofax3.app","title":"Unpack"},{"location":"tutorial/install/#running-eclipse","text":"Start up Eclipse, depending on your operating system: Windows: run Spoofax3/eclipse.exe macOS run Spoofax3.app Linux run Spoofax3/eclipse Warning macOS Sierra (10.12) and above will mark the unpacked Spoofax3.app as \"damaged\" due to a modified signed/notarized application, because we have modified the eclipse.ini file inside it. To fix this, open the Terminal, navigate to the directory where the Spoofax3.app file is located, and execute: xattr -rc Spoofax3.app After starting up, choose where your workspace will be stored. The Eclipse workspace will contain all of your settings, and is the default location for new projects. Warning Currently, there are several bugs regarding spaces in the workspace path, so ensure there are no spaces on the workspace path.","title":"Running Eclipse"},{"location":"tutorial/install/#configuring-eclipses-preferences","text":"Some Eclipse preferences unfortunately have sub-optimal defaults. After you have chosen a workspace and Eclipse has completely started up (and you have closed the Welcome page), go to the Eclipse preferences by pressing Cmd + , on macOS and by going to the Window \u2023 Preferences menu on others, and set these options: General \u2023 Startup and Shutdown Enable: Refresh workspace on startup General \u2023 Workspace Enable: Refresh using native hooks or polling General \u2023 Workspace \u2023 Build Enable: Save automatically before manual build We need to make sure that Eclipse has detected an installed JRE . Open the Eclipse preferences and go to the Java \u2023 Installed JREs page: If there are no installed JREs, and you've downloaded an Eclipse installation with an embedded JVM , press Search... and navigate to the location where you unpacked the Eclipse installation, and choose the jvm directory in it. Then press the checkmark of the JRE to activate it. If there are no installed JREs, and you've downloaded an Eclipse installation without an embedded JVM , press Search... and navigate to the location where your JVM installed, and choose it. Then press the checkmark of the JRE to activate it. If there are one or more installed JVMs, but none are selected, select an appropriate one by pressing the checkmark. If there are one or more installed JVMs, and one is selected, you are good to go. Finally, you may configure Eclipse to your liking. Some typical settings to adjust: General \u2023 Editors \u2023 Text Editors Displayed tab width : change to your desired tab width. Most of Spoofax uses 2 by convention. Insert spaces for tabs : enable, as Spoofax uses spaces by convention. Show print margin : enable and set if you want to set a maximum line length. Show line numbers : enable if you want to see line numbers. General \u2023 Appearance : choose a Theme to your liking. General \u2023 Appearance \u2023 Colors and Fons \u2023 Text Font : the JetBrains Mono font supports ligatures and is used in (most of) the lecture slides (install separately). General \u2023 Keys : change keybindings. Tip These preferences are stored per workspace. If you create a fresh workspace, you have to re-do these settings. You can create a new workspace with copied preferences by selecting File \u2023 Switch workspace \u2023 Other... , and then checking Preferences under Copy settings . Now that Eclipse is set up, continue with creating a language project","title":"Configuring Eclipse's preferences"},{"location":"tutorial/pcf_tutorial/","text":"A Spoofax tutorial implementing Programming Computable Functions (PCF) \u00b6 In computer science, Programming Computable Functions (PCF) is a typed functional language introduced by Gordon Plotkin in 1977, based on previous unpublished material by Dana Scott. It can be considered to be an extended version of the typed lambda calculus or a simplified version of modern typed functional languages such as ML or Haskell. -- https://en.wikipedia.org/wiki/Programming_Computable_Functions With this tutorial you will hopefully be able to define the programming language PCF in the Spoofax language workbench in around one hour. In order to get started quickly, you should clone the Git repository connected to this tutorial. The template directory contains a minimal project setup that allows you to start implementing PCF. Based on this tutorial, you are hopefully able to implement the syntax and static semantics of PCF yourself. In case you get stuck, you can always peek at the example implementation in the implementation directory. Getting started \u00b6 You will need an installation of the Spoofax language workbench, you can find the latest release here . The recommended version to download is the one with embedded JVM. Note that on MacOS and Linux there are some extra instructions after unpacking. You can now import the project following the instructions on the website , which come down to File > Import... ; General > Existing Projects into Workspace ; Next > ; tick Select root directory: ; Browse... to the directory with the spoofaxc.cfg and press Open ; Finish . Now you should Project > Build Project after selecting the project in the Package Explorer . Within the project you will be working on files in the src and test directories. Source material for PCF \u00b6 Since the original publications on PCF were more concerned with semantics than syntax, it is a bit difficult to trace the syntax of PCF. We will use the definition of PCF from a book by John C. Mitchell called \"Foundation for Programming Languages\" (Mitchell 1996). Note that you might also find a definition by Dowek and Levy (Dowek et al. 2011) called Mini-ML, or PCF, this is not the version that we will use here. Chapter 2 of Foundation for Programming Languages about the language PCF is freely available . Grammar \u00b6 This is a slightly massaged grammar of so-called \"pure pcf\" from section 2.2.6: e ::= x (variable reference) | if e then e else e (if condition) | \\x : t. e (function abstraction) | e e (function application) | fix e (fixed point) | true | false (boolean constants) | Eq? e e (equality check) | n (natural number constant) | e + e (arithmetic operations) | (e) (parenthesised expression) t ::= nat (natural number type) | bool (boolean type) | t -> t (function type) | (t) (parenthesised type) As you can see, PCF is a small functional programming language. It is an expression based language where e is the expression sort, and t is the type sort. There are also lexical sorts in this grammar, namely x for names, and n for numeric constants. The other words and symbols are keywords and operators. We've replaced some of the non-ASCII notation from the book into an ASCII version to make it easier to type, but if you have a good input method for non-ascii symbols, feel free to use those in your grammar. We've also added parentheses to both sorts for grouping. We have excluded pairs from the grammar, to make the language a little smaller and hopefully make this tutorial completable in one hour. The grammar in the book is more type-directed, which constrains what programs the parser can parse to already be closer to the set of programs that are actually typed and therefore in the language. While this might seem advantageous, in practice it is nicer for a user to have a wide range of programs parse and be given highlighting. The type checker can give a much clearer explanation of why a program is not acceptable than a parser based on a grammar that encodes some type information. SDF3 \u00b6 With a context-free grammar available to us, we can start implementing the syntax of PCF in Spoofax. For this we use SDF3, the Syntax Definition Formalism version 3. You'll find that the template project already has some .sdf3 files ready for you: lex.sdf3 , expr.sdf , type.sdf3 and main.sdf3 . The main file defines the start symbol of the grammar which is used by the editor to know where to start parsing. Expressions go in expr.sdf3 , types in type.sdf3 and we have already provided you with a lexical syntax in lex.sdf3 . Have a look, you will find the definition of lexical sorts Name , Number and Keyword , where Name and Number are defined as regular expressions, and Keyword is defined as a few options of literal strings that correspond with keywords from the grammar. Then Name is restricted by a rejection rule to not match anything that can be parsed as Keyword . The lexical restrictions make sure that names and numbers are matched greedily. Aside: Grammars in SDF3 define both lexical and context-free syntax, both of which are handled together by a character-level parsing algorithm called SGLR. This makes it hard to provide truly greedy regular expression by default, and instead we express that a name is not allowed to be directly followed by a letter or number, which can be handled better by a parser and still makes things greedy. At the end of the file you find a defining of LAYOUT , which is the white space (and possibly comments) that are allowed between parts of the context-free syntax that we'll specify together for expressions and types. In the files expr.sdf3 and type.sdf3 you will find the sort definitions for expressions and types with some but not all rules. As you can see, there is template syntax in SDF3 which is both a convenient way to write your syntax and a hint for how it might be formatted in a program. Exercise. Try writing the remaining part of the grammar yourself. Once you've built the project again, you can try out the newly added parts of a grammar. In test/test.spt file you will find test written in the SPoofax Testing language SPT . In this special language workbench testing language we can test many things on a high level. For now we can more parse succeeds and parse fails tests and see failing tests get an error marker in the editor immediately. You can also with a parse to test where you can specify the abstract syntax tree you expect. Of course you can also write your PCF programming language in its own editor. There is already an example.pcf file, where you can see the syntax highlighting derived from your grammar. You can also view the abstract syntax tree of this program through the menu Spoofax > Debug > Show parsed AST . The (continuous) version even updates as you update your program. You might find that writing a program 1 + 1 + 1 fails both expectations, because it is in our PCF language, but the parsing isn't entirely successful . Instead the result, which you can also write as an expectation, is parse ambiguous . We need to specify in the grammar what the associativity of the program is, whether it's (1 + 1) + 1 or 1 + (1 + 1) . Let's pick the former and use the {left} annotation on the Expr.Add rule. Now your double-add test should work. In fact we can now use the parse to test to specify that we expect Add(Add(Num(\"1\"), Num(\"1\")), Num(\"1\")) . That is a little cumbersome to write though. What we can also do is write another program between double brackets: [[(1 + 1) + 1]] . Because the round brackets are not in the AST, this comes down to the same test. Now that we're familiar with ambiguities and testing for them, we should root out the other ones in our grammar. You'll find that most grammar productions in PCF are not ambiguous with themselves, but mostly with each other. This is a priority problem, which is specified in a context-free priorities section of the grammar. You can write Expr.App > Expr.Add to specify that application binds tighter than addition. You can write out pairs of these with commas in between, or a longer chain of > , which is more common and is a reminder that priority is transitive. You can also make groups of expressions of the same priority, like { Expr.If Expr.Lam Expr.Fix } . Exercise. See if you can figure out a good set of priorities and write some tests for them. You can check your list against ours in the implementation . Static Semantics \u00b6 PCF has a relatively simple type system, defined as follows: ---------------- [T-True] \u0393 |- true : bool ----------------- [T-False] \u0393 |- false : bool ------------ [T-Nat] \u0393 |- n : nat \u0393 |- e1 : nat \u0393 |- e2 : nat ------------------ [T-Add] \u0393 |- e1 + e2 : nat \u0393 |- e1 : t \u0393 |- e2 : t --------------------- [T-Eq] \u0393 |- Eq? e1 e2 : bool \u0393 |- e : bool \u0393 |- e1 : t \u0393 |- e2 : t ----------------------------- [T-If] \u0393 |- if e then e1 else e2 : t (x, t) \u03f5 \u0393 ---------- [T-Var] \u0393 |- x : t (x, t); \u0393 |- e : t' ------------------------ [T-Abs] \u0393 |- \\x : t. e : t -> t' \u0393 |- e1 : t' -> t \u0393 |- e2 : t' ----------------- [T-App] \u0393 |- e1 e2 : t \u0393 |- e : t -> t --------------- [T-Fix] \u0393 |- fix e : t In this type system, all constructs have monomorphic types. The equality operator accepts any operands, as long as the types of the left and right operand are equal. Similarly, the types of the then-branch and the else-branch of an if-expression should be equal. Variables are typed in a typing environment \u0393, which is extended by lambda abstractions. This is all similar to the regular definition of the lambda calculus (see e.g. Pierce (2002), chap. 9). The typing rule for the fixpoint operator ensures its argument is an endofunction, and types it with the (co)domain of the function. Statix \u00b6 Given this type system and a parser derived from the syntax specification, we can start defining our type-checker in Spoofax. We use the Statix meta-language for type system specification for this. In Statix, type-systems can be expressed in a declarative style, closely related to formal inference rules, such as given above. However, instead of typing environments, scope graphs are used for name binding. (Scope Graphs will be explained in more detail later in this tutorial.) The backend (often referred to as 'solver') interprets the specification applied to an AST of the object language (PCF in this case) as a constraint program, which yields an executable type checker. So let's get started. We will define our type-system in the src/expr.stx file. This file already imports expr-sig and type-sig , which makes the abstract syntax of the language, which is derived from the syntax definition, available. In addition, there is a declaration for a user-defined constraint typeOfExpr , which has type scope * Expr -> Type . That means that the constraint accepts a scope and an expression, and returns a type for it. This can be read as \u0393 |- e : t , where the scope argument takes the role of the environment \u0393. To test the type system, it is recommended to open the example.pcf and test/test.spt files as well. A lonely constraint declaration is useless: there are no ways a typeOfExpr constraint can be solved. We need to add rules for this constraint. Rules can be compared to cases in a functional language or, even better, inference rules as given above. Lets look at an example: typeOfExpr(_, True()) = Bool(). This rule states the simple fact that a true constant has type bool . Do you see the similarity with our T-True rule? Exercise: Define the T-False and T-Nat rules in your Statix specification. That was not too hard. However, not all rules are that simple. Some of them have premises . In Statix, we encode them after a turnstile symbol :- . For example: typeOfExpr(s, Eq(e1, e2)) = Bool() :- {T} typeOfExpr(s, e1) == T, typeOfExpr(s, e2) == T. This rule is a bit more complicated, so lets break it down part by part. The meaning of the rule head ( typeOfExpr(s, Eq(e1, e2)) = Bool() ) should be familiar by now. An equality comparison in scope s has type Bool() . Then, after the turnstile, there is an existential constraint {T} , which introduced the unification variable T . One might read it as \u2203 T. ... . Then there are two premises of the rule that assert that e1 and e2 have the type T . As a unification variable can only have a single value, this effectively enforces both operands to have the same type. Exercise: Define the T-Add and T-If rules. It is also possible to use SPT to test your type system. To apply the type-checker in a test, use the analysis succeeds and analysis fails expectations. For example: test cannot compare nat and bool [[ Eq? true 42 ]] analysis fails Exercise: Define some SPT tests that cover all currently typed constructs. Scope Graphs \u00b6 Now it is time to look add lambda abstractions and applications. In Statix, typing these constructs uses scope graphs. We will explore how scope graphs work using some example programs. The rule for lambda abstraction is already defined as follows: typeOfExpr(s, Lam(x, T, e)) = Fun(T, T') :- {s_lam} new s_lam, s_lam -P-> s, !var[x, T] in s_lam, typeOfExpr(s_lam, e) == T'. In this rule, a lambda expression gets the type Fun(T, T') . The input type T is explicitly provided by the syntax, while the result type T' is inferred from the lambda body. However, as can be seen in the T-Abs rule, the body is typed in an extended context . In Statix, that is encoded by the first three constraints. The new s_lam constraint generates a fresh node in the scope graph, and binds a reference to that node to the unification variable s_lam . That scope encodes the context of the body. To indicate that s_lam inherits all declarations from its parent context (\u0393 in the rule), the s_lam -P-> s constraints asserts an edge from s_lam to s . This ensures that a query in s_lam (explained later) can reach declarations in s . The edge is labeled with an edge label P . In Statix all edges are labeled, and labels have to be introduced explicitly. In the stub, the label P was already predefined in the signature section above the rule for abstractions. Finally, we need to extend the s_lam context with our new variable. This is what the !var[x, T] in s_lam constraint does. This constraint, which is similar to the (x, t); in T-Abs , creates a declaration with name x and type T in s_lam . A declaration always uses a relation , which is var in this case. The var relation is also declared in the signature section above. To aid debugging, scope graph can be inspected. For example, open an example.pcf file, and add this program: (\\double: nat -> nat. double 21) \\x: nat. x + x Now, open the menu Spoofax > Debug > Show formatted scope graph (continuous) . A new window should be opened with (approximately) this content: scope graph #-s_lam_20-4 { relations { expr!var : (\"double\", Fun(Nat(), Nat())) } edges { expr!P : #-s_glob_1-5 } } #-s_lam_10-2 { relations { expr!var : (\"x\", Nat()) } edges { expr!P : #-s_glob_1-5 } } This file shows two scopes: #-s_lam_20-4 and #-s_lam_10-2 . These scopes correspond to the bodies of the first and second abstraction, respectively. Both scopes have a single declaration, shown in the relations block. The contents of the relations should not be surprising: they are the declarations we asserted using our !var[x, T] in s_lam constraint! Similarly, there are two P -labeled edges to #s_glob_1-5 . This scope is the global scope, which does not have its own entry because it is empty. In fact, it corresponds to the empty top-level environment a regular typing derivation would start with. Now, we have seen how to extend a context, but how should we read it? Reading a context corresponds to doing a query in Statix. In PCF, the only time we use a query is when resolving a variable in the T-Var rule. In Statix, this rule is defined as follows: typeOfExpr(s, Var(x)) = T :- {Ts} query var filter P* and { x' :- x' == x } min $ < P in s |-> Ts, referenceTypeOk(x, T, Ts). We ignore the referenceTypeOk constraint for now, as it is only there to ensure the error messages are easier to interpret. The interesting part is the query constraint. This constraint takes quite some parameters, which we will analyse one by one. First, there is the var parameter, which is the relation to query. In languages that, unlike PCF, have multiple relations, this argument ensures that the query will only find declarations under the var relation, such as the ones asserted by the T-Abs rule we've discussed earlier. Second, there is the P* argument. This is a regular expression that describes valid paths in the scope graph. In this case, it ensures that the query can resolve in the local context, and all parent contexts. Exercise. Change P* into P+ and test the program \\x: nat. x . Does it type correctly? Why (not)? Third, there is the data well-formedness condition { x' :- x' == x } . This is an anonymous unary predicate that compares the name of a declaration (bound to x' ) to the reference ( x ). Only declarations for which the constraint holds (i.e., x' == x can be solved) are returned in the query answer. This excludes reachable declarations with the wrong name. Exercise. Change { x' :- x' == x } to { x' :- true } and test the program \\x: nat. \\y: bool. x + 1 . Does it type correctly? Why (not)? Fourth, there is the $ < P argument. This argument indicates that the end-of-path label $ binds closer than the P label. This means that shorter paths are preferred over longer paths, essentially modelling shadowing. This is best seen in action: Exercise. Remove $ < P and test the program \\x: bool. \\x: nat. x + 1 . Does it type correctly? Why (not)? Exercise. Add P < $ and test the program \\x: bool. \\x: nat. x + 1 . Does it type correctly? Why (not)? Exercise: Define the T-App and T-Fix rules. Congratulations! You have now defined a fully functional frontend for PCF. A Small Detour on Editor Services. \u00b6 Type information is often used for editor services and transformations. This is tightly integrated in the Spoofax language as well. For example, hover your mouse over a variable reference for a moment. After some time, a tooltip with (e.g.) the text Type: Nat() will show up. Or, even fancier, CTRL-Click (Cmd-Click on Mac) on a reference. Your cursor now should jump to the binder that introduced the variable. To understand this behaviour, we have to look into the second referenceTypeOk rule. This rule contains the following constraints: @x.type := T, @x.ref := x' In this rule x is the reference, and T is the expected type. The constraint @x.type := T sets the value of the type property of x to T . The Spoofax editor will read such type properties and display them in a tooltip. That explains the tooltip we observed earlier. In addition, the x' variable is the name of the declaration. Unobservable to the user, this name has its position attached. By assigning it as the ref property of the reference x , the editor reference resolution knew where to put the cursor when x was CTRL-clicked! Syntactic Sugar \u00b6 These are the syntactic extensions (syntactic sugar) found in the book we're following: e ::= ... | let x : t = e in e (let binding) | let x(x : t) : t = e in e (let function binding) | letrec x : t = e in e (let recursive binding) | letrec x(x : t) : t = e in e (let recursive binding) These extensions are defined as sugar, in that you can transform them into an equivalent program using only the \"pure pcf\" syntax. That's exactly how we will also implement this sugar. Let's first summarise the meaning of these extensions: let x : t = e1 in e2 == (\\x : t. e2) e1 let x1(x2 : t1) : t2 = e1 in e2 == let x1 : t1 -> t2 = \\x2 : t1. e1 in e2 == (\\x1 : t1 -> t2. e2) (\\x2 : t1. e1) letrec x : t = e1 in e2 == let x : t = fix \\x : t. e1 in e2 == (x : t. e2) (fix \\x : t. e1) letrec x1(x2 : t1) : t2 = e1 in e2 == letrec x1 : t1 -> t2 = \\x2 : t1. e1 in e2 == let x1 : t1 -> t2 = fix \\x1 : t1 -> t2. \\x2 : t1. e1 in e2 == (\\x1 : t1 -> t2. e2) (fix \\x1 : t1 -> t2. \\x2 : t1. e1) As you can see, we can use lambdas for binding variables in a let , and we can use fix for recursive definitions in letrec . Function definition syntax in let is of course also just sugar for lambdas. Stratego 2 \u00b6 Let's turn these equations into executable code. We will use the term-writing language Stratego 2 for this task. You can find code of this language in .str2 files such as main.str2 and desugar.str2 . In the latter you will find a prewritten strategy desugar , defined as a type preserving transformation ( TP ) that traverses topdown and tries to apply the desugar-let rewrite rules. In the rewrite rules is desugar-let , also type preserving, but it currently fails. You can see an example rewrite rule that turns let bound functions into the applications of lambdas from our equations above, but this uses the LetF abstract syntax, which we have not defined yet. So before we start writing our rewrite rules, we should first define the appropriate syntax in expr.sdf3 . Exercise: Define syntax for let and letrec , and don't forget the priority rules of the newly added syntax. With the syntax defined, we can now start writing our rewrite rules. As suggested by the example one for let-bound functions, we are writing the abstract syntax pattern of our sugar programs with variables, then an arrow -> , and then the abstract syntax pattern of our final result. This way we only have to traverse our program once to apply the rules. We could go topdown or bottomup with this approach, either direction works. Exercise: Write the remaining rewrite rules. We could also write our desugaring differently, in the smaller steps of the equation, by repeatedly applying rewrite rules. Our strategy would then be outermost or innermost , and we could rewrite our Let as usual, but write our Let(Rec)F to Let(Rec) and our LetRec to Let with Fix . Exercise: Write the single step rewrite rules. (This is demonstrated in desugar2 of the example implementation .) You can factor out the similarity of the function-binding let and letrec to their normal counterpart. Bonus Exercise: Define new constructors for Let and LetF that take an extra argument that marks if it is a normal or letrec version. Define a separate desugaring step to transform your program into these constructors. Now define the single step rewrite rules from before, combining the similar one into a single rule. (This is demonstrated in desugar3 of the example implementation .) With your desugaring now complete, you can add a call to desugar to pre-analyze in src/main.str2 . This will eliminate the let related constructors before you analyse your program, so you do not need to extend your Statix specification. References \u00b6 (Mitchell 1996) Mitchell, John C. (1996). The Language PCF. In: Foundations for Programming Languages. https://theory.stanford.edu/~jcm/books/fpl-chap2.ps (Dowek et al. 2011) Dowek, G., L\u00e9vy, JJ. (2011). The Language PCF. In: Introduction to the Theory of Programming Languages. Undergraduate Topics in Computer Science. Springer, London. https://doi.org/10.1007/978-0-85729-076-2_2 (Pierce 2002) Pierce, Benjamin C. (2002). Types and Programming Languages. MIT Press, Cambridge, Massachusetts.","title":"A Spoofax tutorial implementing Programming Computable Functions (PCF)"},{"location":"tutorial/pcf_tutorial/#a-spoofax-tutorial-implementing-programming-computable-functions-pcf","text":"In computer science, Programming Computable Functions (PCF) is a typed functional language introduced by Gordon Plotkin in 1977, based on previous unpublished material by Dana Scott. It can be considered to be an extended version of the typed lambda calculus or a simplified version of modern typed functional languages such as ML or Haskell. -- https://en.wikipedia.org/wiki/Programming_Computable_Functions With this tutorial you will hopefully be able to define the programming language PCF in the Spoofax language workbench in around one hour. In order to get started quickly, you should clone the Git repository connected to this tutorial. The template directory contains a minimal project setup that allows you to start implementing PCF. Based on this tutorial, you are hopefully able to implement the syntax and static semantics of PCF yourself. In case you get stuck, you can always peek at the example implementation in the implementation directory.","title":"A Spoofax tutorial implementing Programming Computable Functions (PCF)"},{"location":"tutorial/pcf_tutorial/#getting-started","text":"You will need an installation of the Spoofax language workbench, you can find the latest release here . The recommended version to download is the one with embedded JVM. Note that on MacOS and Linux there are some extra instructions after unpacking. You can now import the project following the instructions on the website , which come down to File > Import... ; General > Existing Projects into Workspace ; Next > ; tick Select root directory: ; Browse... to the directory with the spoofaxc.cfg and press Open ; Finish . Now you should Project > Build Project after selecting the project in the Package Explorer . Within the project you will be working on files in the src and test directories.","title":"Getting started"},{"location":"tutorial/pcf_tutorial/#source-material-for-pcf","text":"Since the original publications on PCF were more concerned with semantics than syntax, it is a bit difficult to trace the syntax of PCF. We will use the definition of PCF from a book by John C. Mitchell called \"Foundation for Programming Languages\" (Mitchell 1996). Note that you might also find a definition by Dowek and Levy (Dowek et al. 2011) called Mini-ML, or PCF, this is not the version that we will use here. Chapter 2 of Foundation for Programming Languages about the language PCF is freely available .","title":"Source material for PCF"},{"location":"tutorial/pcf_tutorial/#grammar","text":"This is a slightly massaged grammar of so-called \"pure pcf\" from section 2.2.6: e ::= x (variable reference) | if e then e else e (if condition) | \\x : t. e (function abstraction) | e e (function application) | fix e (fixed point) | true | false (boolean constants) | Eq? e e (equality check) | n (natural number constant) | e + e (arithmetic operations) | (e) (parenthesised expression) t ::= nat (natural number type) | bool (boolean type) | t -> t (function type) | (t) (parenthesised type) As you can see, PCF is a small functional programming language. It is an expression based language where e is the expression sort, and t is the type sort. There are also lexical sorts in this grammar, namely x for names, and n for numeric constants. The other words and symbols are keywords and operators. We've replaced some of the non-ASCII notation from the book into an ASCII version to make it easier to type, but if you have a good input method for non-ascii symbols, feel free to use those in your grammar. We've also added parentheses to both sorts for grouping. We have excluded pairs from the grammar, to make the language a little smaller and hopefully make this tutorial completable in one hour. The grammar in the book is more type-directed, which constrains what programs the parser can parse to already be closer to the set of programs that are actually typed and therefore in the language. While this might seem advantageous, in practice it is nicer for a user to have a wide range of programs parse and be given highlighting. The type checker can give a much clearer explanation of why a program is not acceptable than a parser based on a grammar that encodes some type information.","title":"Grammar"},{"location":"tutorial/pcf_tutorial/#sdf3","text":"With a context-free grammar available to us, we can start implementing the syntax of PCF in Spoofax. For this we use SDF3, the Syntax Definition Formalism version 3. You'll find that the template project already has some .sdf3 files ready for you: lex.sdf3 , expr.sdf , type.sdf3 and main.sdf3 . The main file defines the start symbol of the grammar which is used by the editor to know where to start parsing. Expressions go in expr.sdf3 , types in type.sdf3 and we have already provided you with a lexical syntax in lex.sdf3 . Have a look, you will find the definition of lexical sorts Name , Number and Keyword , where Name and Number are defined as regular expressions, and Keyword is defined as a few options of literal strings that correspond with keywords from the grammar. Then Name is restricted by a rejection rule to not match anything that can be parsed as Keyword . The lexical restrictions make sure that names and numbers are matched greedily. Aside: Grammars in SDF3 define both lexical and context-free syntax, both of which are handled together by a character-level parsing algorithm called SGLR. This makes it hard to provide truly greedy regular expression by default, and instead we express that a name is not allowed to be directly followed by a letter or number, which can be handled better by a parser and still makes things greedy. At the end of the file you find a defining of LAYOUT , which is the white space (and possibly comments) that are allowed between parts of the context-free syntax that we'll specify together for expressions and types. In the files expr.sdf3 and type.sdf3 you will find the sort definitions for expressions and types with some but not all rules. As you can see, there is template syntax in SDF3 which is both a convenient way to write your syntax and a hint for how it might be formatted in a program. Exercise. Try writing the remaining part of the grammar yourself. Once you've built the project again, you can try out the newly added parts of a grammar. In test/test.spt file you will find test written in the SPoofax Testing language SPT . In this special language workbench testing language we can test many things on a high level. For now we can more parse succeeds and parse fails tests and see failing tests get an error marker in the editor immediately. You can also with a parse to test where you can specify the abstract syntax tree you expect. Of course you can also write your PCF programming language in its own editor. There is already an example.pcf file, where you can see the syntax highlighting derived from your grammar. You can also view the abstract syntax tree of this program through the menu Spoofax > Debug > Show parsed AST . The (continuous) version even updates as you update your program. You might find that writing a program 1 + 1 + 1 fails both expectations, because it is in our PCF language, but the parsing isn't entirely successful . Instead the result, which you can also write as an expectation, is parse ambiguous . We need to specify in the grammar what the associativity of the program is, whether it's (1 + 1) + 1 or 1 + (1 + 1) . Let's pick the former and use the {left} annotation on the Expr.Add rule. Now your double-add test should work. In fact we can now use the parse to test to specify that we expect Add(Add(Num(\"1\"), Num(\"1\")), Num(\"1\")) . That is a little cumbersome to write though. What we can also do is write another program between double brackets: [[(1 + 1) + 1]] . Because the round brackets are not in the AST, this comes down to the same test. Now that we're familiar with ambiguities and testing for them, we should root out the other ones in our grammar. You'll find that most grammar productions in PCF are not ambiguous with themselves, but mostly with each other. This is a priority problem, which is specified in a context-free priorities section of the grammar. You can write Expr.App > Expr.Add to specify that application binds tighter than addition. You can write out pairs of these with commas in between, or a longer chain of > , which is more common and is a reminder that priority is transitive. You can also make groups of expressions of the same priority, like { Expr.If Expr.Lam Expr.Fix } . Exercise. See if you can figure out a good set of priorities and write some tests for them. You can check your list against ours in the implementation .","title":"SDF3"},{"location":"tutorial/pcf_tutorial/#static-semantics","text":"PCF has a relatively simple type system, defined as follows: ---------------- [T-True] \u0393 |- true : bool ----------------- [T-False] \u0393 |- false : bool ------------ [T-Nat] \u0393 |- n : nat \u0393 |- e1 : nat \u0393 |- e2 : nat ------------------ [T-Add] \u0393 |- e1 + e2 : nat \u0393 |- e1 : t \u0393 |- e2 : t --------------------- [T-Eq] \u0393 |- Eq? e1 e2 : bool \u0393 |- e : bool \u0393 |- e1 : t \u0393 |- e2 : t ----------------------------- [T-If] \u0393 |- if e then e1 else e2 : t (x, t) \u03f5 \u0393 ---------- [T-Var] \u0393 |- x : t (x, t); \u0393 |- e : t' ------------------------ [T-Abs] \u0393 |- \\x : t. e : t -> t' \u0393 |- e1 : t' -> t \u0393 |- e2 : t' ----------------- [T-App] \u0393 |- e1 e2 : t \u0393 |- e : t -> t --------------- [T-Fix] \u0393 |- fix e : t In this type system, all constructs have monomorphic types. The equality operator accepts any operands, as long as the types of the left and right operand are equal. Similarly, the types of the then-branch and the else-branch of an if-expression should be equal. Variables are typed in a typing environment \u0393, which is extended by lambda abstractions. This is all similar to the regular definition of the lambda calculus (see e.g. Pierce (2002), chap. 9). The typing rule for the fixpoint operator ensures its argument is an endofunction, and types it with the (co)domain of the function.","title":"Static Semantics"},{"location":"tutorial/pcf_tutorial/#statix","text":"Given this type system and a parser derived from the syntax specification, we can start defining our type-checker in Spoofax. We use the Statix meta-language for type system specification for this. In Statix, type-systems can be expressed in a declarative style, closely related to formal inference rules, such as given above. However, instead of typing environments, scope graphs are used for name binding. (Scope Graphs will be explained in more detail later in this tutorial.) The backend (often referred to as 'solver') interprets the specification applied to an AST of the object language (PCF in this case) as a constraint program, which yields an executable type checker. So let's get started. We will define our type-system in the src/expr.stx file. This file already imports expr-sig and type-sig , which makes the abstract syntax of the language, which is derived from the syntax definition, available. In addition, there is a declaration for a user-defined constraint typeOfExpr , which has type scope * Expr -> Type . That means that the constraint accepts a scope and an expression, and returns a type for it. This can be read as \u0393 |- e : t , where the scope argument takes the role of the environment \u0393. To test the type system, it is recommended to open the example.pcf and test/test.spt files as well. A lonely constraint declaration is useless: there are no ways a typeOfExpr constraint can be solved. We need to add rules for this constraint. Rules can be compared to cases in a functional language or, even better, inference rules as given above. Lets look at an example: typeOfExpr(_, True()) = Bool(). This rule states the simple fact that a true constant has type bool . Do you see the similarity with our T-True rule? Exercise: Define the T-False and T-Nat rules in your Statix specification. That was not too hard. However, not all rules are that simple. Some of them have premises . In Statix, we encode them after a turnstile symbol :- . For example: typeOfExpr(s, Eq(e1, e2)) = Bool() :- {T} typeOfExpr(s, e1) == T, typeOfExpr(s, e2) == T. This rule is a bit more complicated, so lets break it down part by part. The meaning of the rule head ( typeOfExpr(s, Eq(e1, e2)) = Bool() ) should be familiar by now. An equality comparison in scope s has type Bool() . Then, after the turnstile, there is an existential constraint {T} , which introduced the unification variable T . One might read it as \u2203 T. ... . Then there are two premises of the rule that assert that e1 and e2 have the type T . As a unification variable can only have a single value, this effectively enforces both operands to have the same type. Exercise: Define the T-Add and T-If rules. It is also possible to use SPT to test your type system. To apply the type-checker in a test, use the analysis succeeds and analysis fails expectations. For example: test cannot compare nat and bool [[ Eq? true 42 ]] analysis fails Exercise: Define some SPT tests that cover all currently typed constructs.","title":"Statix"},{"location":"tutorial/pcf_tutorial/#scope-graphs","text":"Now it is time to look add lambda abstractions and applications. In Statix, typing these constructs uses scope graphs. We will explore how scope graphs work using some example programs. The rule for lambda abstraction is already defined as follows: typeOfExpr(s, Lam(x, T, e)) = Fun(T, T') :- {s_lam} new s_lam, s_lam -P-> s, !var[x, T] in s_lam, typeOfExpr(s_lam, e) == T'. In this rule, a lambda expression gets the type Fun(T, T') . The input type T is explicitly provided by the syntax, while the result type T' is inferred from the lambda body. However, as can be seen in the T-Abs rule, the body is typed in an extended context . In Statix, that is encoded by the first three constraints. The new s_lam constraint generates a fresh node in the scope graph, and binds a reference to that node to the unification variable s_lam . That scope encodes the context of the body. To indicate that s_lam inherits all declarations from its parent context (\u0393 in the rule), the s_lam -P-> s constraints asserts an edge from s_lam to s . This ensures that a query in s_lam (explained later) can reach declarations in s . The edge is labeled with an edge label P . In Statix all edges are labeled, and labels have to be introduced explicitly. In the stub, the label P was already predefined in the signature section above the rule for abstractions. Finally, we need to extend the s_lam context with our new variable. This is what the !var[x, T] in s_lam constraint does. This constraint, which is similar to the (x, t); in T-Abs , creates a declaration with name x and type T in s_lam . A declaration always uses a relation , which is var in this case. The var relation is also declared in the signature section above. To aid debugging, scope graph can be inspected. For example, open an example.pcf file, and add this program: (\\double: nat -> nat. double 21) \\x: nat. x + x Now, open the menu Spoofax > Debug > Show formatted scope graph (continuous) . A new window should be opened with (approximately) this content: scope graph #-s_lam_20-4 { relations { expr!var : (\"double\", Fun(Nat(), Nat())) } edges { expr!P : #-s_glob_1-5 } } #-s_lam_10-2 { relations { expr!var : (\"x\", Nat()) } edges { expr!P : #-s_glob_1-5 } } This file shows two scopes: #-s_lam_20-4 and #-s_lam_10-2 . These scopes correspond to the bodies of the first and second abstraction, respectively. Both scopes have a single declaration, shown in the relations block. The contents of the relations should not be surprising: they are the declarations we asserted using our !var[x, T] in s_lam constraint! Similarly, there are two P -labeled edges to #s_glob_1-5 . This scope is the global scope, which does not have its own entry because it is empty. In fact, it corresponds to the empty top-level environment a regular typing derivation would start with. Now, we have seen how to extend a context, but how should we read it? Reading a context corresponds to doing a query in Statix. In PCF, the only time we use a query is when resolving a variable in the T-Var rule. In Statix, this rule is defined as follows: typeOfExpr(s, Var(x)) = T :- {Ts} query var filter P* and { x' :- x' == x } min $ < P in s |-> Ts, referenceTypeOk(x, T, Ts). We ignore the referenceTypeOk constraint for now, as it is only there to ensure the error messages are easier to interpret. The interesting part is the query constraint. This constraint takes quite some parameters, which we will analyse one by one. First, there is the var parameter, which is the relation to query. In languages that, unlike PCF, have multiple relations, this argument ensures that the query will only find declarations under the var relation, such as the ones asserted by the T-Abs rule we've discussed earlier. Second, there is the P* argument. This is a regular expression that describes valid paths in the scope graph. In this case, it ensures that the query can resolve in the local context, and all parent contexts. Exercise. Change P* into P+ and test the program \\x: nat. x . Does it type correctly? Why (not)? Third, there is the data well-formedness condition { x' :- x' == x } . This is an anonymous unary predicate that compares the name of a declaration (bound to x' ) to the reference ( x ). Only declarations for which the constraint holds (i.e., x' == x can be solved) are returned in the query answer. This excludes reachable declarations with the wrong name. Exercise. Change { x' :- x' == x } to { x' :- true } and test the program \\x: nat. \\y: bool. x + 1 . Does it type correctly? Why (not)? Fourth, there is the $ < P argument. This argument indicates that the end-of-path label $ binds closer than the P label. This means that shorter paths are preferred over longer paths, essentially modelling shadowing. This is best seen in action: Exercise. Remove $ < P and test the program \\x: bool. \\x: nat. x + 1 . Does it type correctly? Why (not)? Exercise. Add P < $ and test the program \\x: bool. \\x: nat. x + 1 . Does it type correctly? Why (not)? Exercise: Define the T-App and T-Fix rules. Congratulations! You have now defined a fully functional frontend for PCF.","title":"Scope Graphs"},{"location":"tutorial/pcf_tutorial/#a-small-detour-on-editor-services","text":"Type information is often used for editor services and transformations. This is tightly integrated in the Spoofax language as well. For example, hover your mouse over a variable reference for a moment. After some time, a tooltip with (e.g.) the text Type: Nat() will show up. Or, even fancier, CTRL-Click (Cmd-Click on Mac) on a reference. Your cursor now should jump to the binder that introduced the variable. To understand this behaviour, we have to look into the second referenceTypeOk rule. This rule contains the following constraints: @x.type := T, @x.ref := x' In this rule x is the reference, and T is the expected type. The constraint @x.type := T sets the value of the type property of x to T . The Spoofax editor will read such type properties and display them in a tooltip. That explains the tooltip we observed earlier. In addition, the x' variable is the name of the declaration. Unobservable to the user, this name has its position attached. By assigning it as the ref property of the reference x , the editor reference resolution knew where to put the cursor when x was CTRL-clicked!","title":"A Small Detour on Editor Services."},{"location":"tutorial/pcf_tutorial/#syntactic-sugar","text":"These are the syntactic extensions (syntactic sugar) found in the book we're following: e ::= ... | let x : t = e in e (let binding) | let x(x : t) : t = e in e (let function binding) | letrec x : t = e in e (let recursive binding) | letrec x(x : t) : t = e in e (let recursive binding) These extensions are defined as sugar, in that you can transform them into an equivalent program using only the \"pure pcf\" syntax. That's exactly how we will also implement this sugar. Let's first summarise the meaning of these extensions: let x : t = e1 in e2 == (\\x : t. e2) e1 let x1(x2 : t1) : t2 = e1 in e2 == let x1 : t1 -> t2 = \\x2 : t1. e1 in e2 == (\\x1 : t1 -> t2. e2) (\\x2 : t1. e1) letrec x : t = e1 in e2 == let x : t = fix \\x : t. e1 in e2 == (x : t. e2) (fix \\x : t. e1) letrec x1(x2 : t1) : t2 = e1 in e2 == letrec x1 : t1 -> t2 = \\x2 : t1. e1 in e2 == let x1 : t1 -> t2 = fix \\x1 : t1 -> t2. \\x2 : t1. e1 in e2 == (\\x1 : t1 -> t2. e2) (fix \\x1 : t1 -> t2. \\x2 : t1. e1) As you can see, we can use lambdas for binding variables in a let , and we can use fix for recursive definitions in letrec . Function definition syntax in let is of course also just sugar for lambdas.","title":"Syntactic Sugar"},{"location":"tutorial/pcf_tutorial/#stratego-2","text":"Let's turn these equations into executable code. We will use the term-writing language Stratego 2 for this task. You can find code of this language in .str2 files such as main.str2 and desugar.str2 . In the latter you will find a prewritten strategy desugar , defined as a type preserving transformation ( TP ) that traverses topdown and tries to apply the desugar-let rewrite rules. In the rewrite rules is desugar-let , also type preserving, but it currently fails. You can see an example rewrite rule that turns let bound functions into the applications of lambdas from our equations above, but this uses the LetF abstract syntax, which we have not defined yet. So before we start writing our rewrite rules, we should first define the appropriate syntax in expr.sdf3 . Exercise: Define syntax for let and letrec , and don't forget the priority rules of the newly added syntax. With the syntax defined, we can now start writing our rewrite rules. As suggested by the example one for let-bound functions, we are writing the abstract syntax pattern of our sugar programs with variables, then an arrow -> , and then the abstract syntax pattern of our final result. This way we only have to traverse our program once to apply the rules. We could go topdown or bottomup with this approach, either direction works. Exercise: Write the remaining rewrite rules. We could also write our desugaring differently, in the smaller steps of the equation, by repeatedly applying rewrite rules. Our strategy would then be outermost or innermost , and we could rewrite our Let as usual, but write our Let(Rec)F to Let(Rec) and our LetRec to Let with Fix . Exercise: Write the single step rewrite rules. (This is demonstrated in desugar2 of the example implementation .) You can factor out the similarity of the function-binding let and letrec to their normal counterpart. Bonus Exercise: Define new constructors for Let and LetF that take an extra argument that marks if it is a normal or letrec version. Define a separate desugaring step to transform your program into these constructors. Now define the single step rewrite rules from before, combining the similar one into a single rule. (This is demonstrated in desugar3 of the example implementation .) With your desugaring now complete, you can add a call to desugar to pre-analyze in src/main.str2 . This will eliminate the let related constructors before you analyse your program, so you do not need to extend your Statix specification.","title":"Stratego 2"},{"location":"tutorial/pcf_tutorial/#references","text":"(Mitchell 1996) Mitchell, John C. (1996). The Language PCF. In: Foundations for Programming Languages. https://theory.stanford.edu/~jcm/books/fpl-chap2.ps (Dowek et al. 2011) Dowek, G., L\u00e9vy, JJ. (2011). The Language PCF. In: Introduction to the Theory of Programming Languages. Undergraduate Topics in Computer Science. Springer, London. https://doi.org/10.1007/978-0-85729-076-2_2 (Pierce 2002) Pierce, Benjamin C. (2002). Types and Programming Languages. MIT Press, Cambridge, Massachusetts.","title":"References"}]} \ No newline at end of file +{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Spoofax 3 \u00b6 Spoofax 3 is a modular and incremental textual language workbench running on the JVM : a collection of tools and Java libraries that enable the development of textual languages, embeddable into batch compilers, code editors and IDEs, or custom applications. It is a reimplementation of Spoofax 2 , with the goal of being more modular, flexible, and correctly incremental. This documentation website aims to cover language development with Spoofax 3, development of the language workbench itself (e.g., meta-language development), and development of this documentation. Currently, Spoofax 3 is experimental and still a work-in-progress. Therefore, it does not have a stable API, lacks documentation and test coverage, and has not yet been applied to real-world use cases. If you are looking for a more mature alternative, see Spoofax 2 , which Spoofax 3 is based on. The documentation for Spoofax 3 is split into five parts: Tutorials : hands-on tutorials to getting you started with Spoofax 3. How-to guides : step-by-step guides showing how to implement certain features or how to solve common problems. Reference : technical reference describing the various facets of Spoofax 3. Background : background information and discussion on various key topics. Releases : download links, changelogs, and migration guides.","title":"Home"},{"location":"#spoofax-3","text":"Spoofax 3 is a modular and incremental textual language workbench running on the JVM : a collection of tools and Java libraries that enable the development of textual languages, embeddable into batch compilers, code editors and IDEs, or custom applications. It is a reimplementation of Spoofax 2 , with the goal of being more modular, flexible, and correctly incremental. This documentation website aims to cover language development with Spoofax 3, development of the language workbench itself (e.g., meta-language development), and development of this documentation. Currently, Spoofax 3 is experimental and still a work-in-progress. Therefore, it does not have a stable API, lacks documentation and test coverage, and has not yet been applied to real-world use cases. If you are looking for a more mature alternative, see Spoofax 2 , which Spoofax 3 is based on. The documentation for Spoofax 3 is split into five parts: Tutorials : hands-on tutorials to getting you started with Spoofax 3. How-to guides : step-by-step guides showing how to implement certain features or how to solve common problems. Reference : technical reference describing the various facets of Spoofax 3. Background : background information and discussion on various key topics. Releases : download links, changelogs, and migration guides.","title":"Spoofax 3"},{"location":"info/","text":"Macros Plugin Environment \u00b6 General List \u00b6 All available variables and filters within the macros plugin: Variable Type Content extra dict config MkDocsConfig {'config_file_path': '/home/runner/work/spoofax-pie/spoofax-pie/mkdocs.yml', 'site_name': 'Spoofax 3', 'nav': [{'Home': 'index.md'}, {'Tutorials': ['tutorial/install.md', 'tutorial/create_language_project.md', 'tutorial/change_static_semantics.md', 'tutorial/add_transformation.md', 'tutorial/pcf_tutorial.md']}, {'How-to Guides': ['guide/ask_for_help.md', 'guide/report_a_bug.md', {'Eclipse LWB': ['guide/eclipse_lwb/import.md', 'guide/eclipse_lwb/update.md', 'guide/eclipse_lwb/troubleshooting.md']}, {'Static Semantics': ['guide/static-semantics/code-completion.md']}, {'Development': ['guide/development/debugging-in-intellij.md', 'guide/development/troubleshooting.md']}]}, {'Reference': ['reference/configuration.md', 'reference/eclipse-lwb/eclipse-project-files.md']}, {'Background': ['background/documentation.md', 'background/motivation.md', 'background/key_ideas.md', 'background/status.md']}, {'Releases': ['release/download.md']}], 'pages': None, 'site_url': None, 'site_description': 'Spoofax 3 documentation website', 'site_author': None, 'theme': Theme(name='material', dirs=['/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/material', '/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''), language='en', direction=None, features=['navigation.expand', 'navigation.instant', 'navigation.tabs', 'navigation.tabs.sticky', 'navigation.sections', 'navigation.indexes', 'navigation.top'], palette={'primary': 'indigo', 'accent': 'indigo'}, font={'code': 'JetBrains Mono'}, icon=None, favicon='assets/spoofax.png', include_search_page=False, search_index_only=True, logo='assets/spoofax.svg'), 'docs_dir': '/home/runner/work/spoofax-pie/spoofax-pie/docs', 'site_dir': '/home/runner/work/spoofax-pie/spoofax-pie/site', 'copyright': None, 'google_analytics': None, 'dev_addr': _IpAddressValue(host='127.0.0.1', port=8000), 'use_directory_urls': True, 'repo_url': 'https://github.com/metaborg/spoofax-pie', 'repo_name': 'metaborg/spoofax-pie', 'edit_uri_template': None, 'edit_uri': 'edit/master/docs/', 'extra_css': ['extra.css'], 'extra_javascript': [], 'extra_templates': [], 'markdown_extensions': ['toc', 'tables', 'fenced_code', 'abbr', 'admonition', 'attr_list', 'codehilite', 'def_list', 'pymdownx.betterem', 'pymdownx.caret', 'pymdownx.details', 'pymdownx.emoji', 'pymdownx.inlinehilite', 'pymdownx.keys', 'pymdownx.magiclink', 'pymdownx.mark', 'pymdownx.saneheaders', 'pymdownx.smartsymbols', 'pymdownx.snippets', 'pymdownx.superfences', 'pymdownx.tabbed', 'pymdownx.tasklist', 'pymdownx.tilde'], 'mdx_configs': {'codehilite': {'guess_lang': False}, 'toc': {'permalink': True}, 'pymdownx.betterem': {'smart_enable': 'all'}, 'pymdownx.emoji': {'emoji_index': , 'emoji_generator': }, 'pymdownx.tasklist': {'custom_checkbox': True}}, 'strict': False, 'remote_branch': 'gh-pages', 'remote_name': 'origin', 'extra': {}, 'plugins': PluginCollection([('search', ), ('macros', ), ('git-revision-date', )]), 'hooks': {}, 'watch': []} environment dict system = 'Linux', system_version = '6.5.0-1024-azure', python_version = '3.8.18', mkdocs_version = '1.4.0', macros_plugin_version = '0.7.0', jinja2_version = '3.1.4' plugin LegacyConfig {'module_name': 'docs/macro', 'modules': [], 'include_dir': 'docs/include', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '', 'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False} git dict status = True, date [ datetime ], short_commit = 'aec1bcb', commit = 'aec1bcbef07cce68c7411506b99132762273a448', tag = '', author = 'Jeff Smits', author_email = 'j.smits-1@tudelft.nl', committer = 'Jeff Smits', committer_email = 'j.smits-1@tudelft.nl', date_ISO = 'Mon Jul 29 16:25:35 2024 +0200', message = 'nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\\n\\nThis only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects', raw = 'commit aec1bcbef07cce68c7411506b99132762273a448\\nAuthor: Jeff Smits \\nDate: Mon Jul 29 16:25:32 2024 +0200\\n\\n nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\\n \\n This only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects', root_dir = '/home/runner/work/spoofax-pie/spoofax-pie' os dict windows = ':fontawesome-brands-windows: Windows', linux = ':fontawesome-brands-linux: Linux', macos = ':fontawesome-brands-apple: macOS' release dict 0.22.0 [ dict ], 0.20.0 [ dict ], 0.19.8 [ dict ], 0.19.7 [ dict ], 0.19.4 [ dict ], 0.19.3 [ dict ], 0.19.2 [ dict ], 0.19.1 [ dict ], 0.19.0 [ dict ], 0.18.0 [ dict ], 0.17.0 [ dict ], 0.16.17 [ dict ], 0.16.16 [ dict ], 0.16.15 [ dict ], 0.16.14 [ dict ], 0.16.13 [ dict ], 0.16.12 [ dict ], 0.16.11 [ dict ], 0.16.10 [ dict ], 0.16.9 [ dict ], 0.16.8 [ dict ], 0.16.7 [ dict ], 0.16.6 [ dict ], 0.16.5 [ dict ], 0.16.4 [ dict ], 0.16.3 [ dict ], 0.16.2 [ dict ], 0.16.1 [ dict ], 0.16.0 [ dict ], 0.15.3 [ dict ], 0.15.2 [ dict ], 0.15.1 [ dict ], 0.15.0 [ dict ], 0.14.2 [ dict ], 0.14.1 [ dict ], 0.14.0 [ dict ], 0.13.0 [ dict ], 0.12.1 [ dict ], 0.12.0 [ dict ], 0.11.13 [ dict ], 0.11.12 [ dict ], 0.11.11 [ dict ], 0.11.10 [ dict ], 0.11.9 [ dict ], 0.11.8 [ dict ], 0.11.7 [ dict ], 0.11.6 [ dict ], 0.11.5 [ dict ], 0.11.4 [ dict ], 0.11.3 [ dict ], 0.11.2 [ dict ], 0.11.1 [ dict ], 0.11.0 [ dict ], 0.10.0 [ dict ], 0.9.0 [ dict ], 0.8.0 [ dict ], rel [ dict ], dev [ dict ] macros SuperDict context [ function ], macros_info [ function ], now [ function ], fix_url [ function ] filters dict pretty [ function ] filters_builtin dict abs [ builtin_function_or_method ], attr [ function ], batch [ function ], capitalize [ function ], center [ function ], count [ builtin_function_or_method ], d [ function ], default [ function ], dictsort [ function ], e [ builtin_function_or_method ], escape [ builtin_function_or_method ], filesizeformat [ function ], first [ function ], float [ function ], forceescape [ function ], format [ function ], groupby [ function ], indent [ function ], int [ function ], join [ function ], last [ function ], length [ builtin_function_or_method ], list [ function ], lower [ function ], items [ function ], map [ function ], min [ function ], max [ function ], pprint [ function ], random [ function ], reject [ function ], rejectattr [ function ], replace [ function ], reverse [ function ], round [ function ], safe [ function ], select [ function ], selectattr [ function ], slice [ function ], sort [ function ], string [ builtin_function_or_method ], striptags [ function ], sum [ function ], title [ function ], trim [ function ], truncate [ function ], unique [ function ], upper [ function ], urlencode [ function ], urlize [ function ], wordcount [ function ], wordwrap [ function ], xmlattr [ function ], tojson [ function ] navigation Navigation Page(title='Home', url='.') Section(title='Tutorials') Page(title=[blank], url='tutorial/install/') Page(title=[blank], url='tutorial/create_language_project/') Page(title=[blank], url='tutorial/change_static_semantics/') Page(title=[blank], url='tutorial/add_transformation/') Page(title=[blank], url='tutorial/pcf_tutorial/') Section(title='How-to Guides') Page(title=[blank], url='guide/ask_for_help/') Page(title=[blank], url='guide/report_a_bug/') Section(title='Eclipse LWB') Page(title=[blank], url='guide/eclipse_lwb/import/') Page(title=[blank], url='guide/eclipse_lwb/update/') Page(title=[blank], url='guide/eclipse_lwb/troubleshooting/') Section(title='Static Semantics') Page(title=[blank], url='guide/static-semantics/code-completion/') Section(title='Development') Page(title=[blank], url='guide/development/debugging-in-intellij/') Page(title=[blank], url='guide/development/troubleshooting/') Section(title='Reference') Page(title=[blank], url='reference/configuration/') Page(title=[blank], url='reference/eclipse-lwb/eclipse-project-files/') Section(title='Background') Page(title=[blank], url='background/documentation/') Page(title=[blank], url='background/motivation/') Page(title=[blank], url='background/key_ideas/') Page(title=[blank], url='background/status/') Section(title='Releases') Page(title=[blank], url='release/download/') files Files page Page Page(title='Info', url='info/') Config Information \u00b6 Standard MkDocs configuration information. Do not try to modify. e.g. {{ config.docs_dir }} See also the MkDocs documentation on the config object . Variable Type Content config_file_path str '/home/runner/work/spoofax-pie/spoofax-pie/mkdocs.yml' site_name str 'Spoofax 3' nav list [{'Home': 'index.md'}, {'Tutorials': ['tutorial/install.md', 'tutorial/create_language_project.md', 'tutorial/change_static_semantics.md', 'tutorial/add_transformation.md', 'tutorial/pcf_tutorial.md']}, {'How-to Guides': ['guide/ask_for_help.md', 'guide/report_a_bug.md', {'Eclipse LWB': ['guide/eclipse_lwb/import.md', 'guide/eclipse_lwb/update.md', 'guide/eclipse_lwb/troubleshooting.md']}, {'Static Semantics': ['guide/static-semantics/code-completion.md']}, {'Development': ['guide/development/debugging-in-intellij.md', 'guide/development/troubleshooting.md']}]}, {'Reference': ['reference/configuration.md', 'reference/eclipse-lwb/eclipse-project-files.md']}, {'Background': ['background/documentation.md', 'background/motivation.md', 'background/key_ideas.md', 'background/status.md']}, {'Releases': ['release/download.md']}] pages NoneType None site_url NoneType None site_description str 'Spoofax 3 documentation website' site_author NoneType None theme Theme Theme(name='material', dirs=['/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/material', '/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''), language='en', direction=None, features=['navigation.expand', 'navigation.instant', 'navigation.tabs', 'navigation.tabs.sticky', 'navigation.sections', 'navigation.indexes', 'navigation.top'], palette={'primary': 'indigo', 'accent': 'indigo'}, font={'code': 'JetBrains Mono'}, icon=None, favicon='assets/spoofax.png', include_search_page=False, search_index_only=True, logo='assets/spoofax.svg') docs_dir str '/home/runner/work/spoofax-pie/spoofax-pie/docs' site_dir str '/home/runner/work/spoofax-pie/spoofax-pie/site' copyright NoneType None google_analytics NoneType None dev_addr _IpAddressValue _IpAddressValue(host='127.0.0.1', port=8000) use_directory_urls bool True repo_url str 'https://github.com/metaborg/spoofax-pie' repo_name str 'metaborg/spoofax-pie' edit_uri_template NoneType None edit_uri str 'edit/master/docs/' extra_css list ['extra.css'] extra_javascript list [] extra_templates list [] markdown_extensions list ['toc', 'tables', 'fenced_code', 'abbr', 'admonition', 'attr_list', 'codehilite', 'def_list', 'pymdownx.betterem', 'pymdownx.caret', 'pymdownx.details', 'pymdownx.emoji', 'pymdownx.inlinehilite', 'pymdownx.keys', 'pymdownx.magiclink', 'pymdownx.mark', 'pymdownx.saneheaders', 'pymdownx.smartsymbols', 'pymdownx.snippets', 'pymdownx.superfences', 'pymdownx.tabbed', 'pymdownx.tasklist', 'pymdownx.tilde'] mdx_configs dict codehilite [ dict ], toc [ dict ], pymdownx.betterem [ dict ], pymdownx.emoji [ dict ], pymdownx.tasklist [ dict ] strict bool False remote_branch str 'gh-pages' remote_name str 'origin' extra LegacyConfig {} plugins PluginCollection search [ SearchPlugin ], macros [ MacrosPlugin ], git-revision-date [ GitRevisionDatePlugin ] hooks dict watch list [] Macros \u00b6 These macros have been defined programmatically for this environment (module or pluglets). Variable Type Content context function ( obj, e ) Default mkdocs_macro List the defined variables macros_info function ( ) Test/debug function: list useful documentation on the mkdocs_macro environment. now function ( ) Get the current time (returns a datetime object). Used alone, it provides a timestamp. To get the year use now().year , for the month number now().month , etc. fix_url function ( url, r ) If url is relative, fix it so that it points to the docs diretory. This is necessary because relative links in markdown must be adapted in html ('img/foo.png' => '../img/img.png'). Git Information \u00b6 Information available on the last commit and the git repository containing the documentation project: e.g. {{ git.message }} Variable Type Content status bool True date datetime datetime.datetime(2024, 7, 29, 16, 25, 35, tzinfo=tzoffset(None, 7200)) short_commit str 'aec1bcb' commit str 'aec1bcbef07cce68c7411506b99132762273a448' tag str '' author str 'Jeff Smits' author_email str 'j.smits-1@tudelft.nl' committer str 'Jeff Smits' committer_email str 'j.smits-1@tudelft.nl' date_ISO str 'Mon Jul 29 16:25:35 2024 +0200' message str 'nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\\n\\nThis only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects' raw str 'commit aec1bcbef07cce68c7411506b99132762273a448\\nAuthor: Jeff Smits \\nDate: Mon Jul 29 16:25:32 2024 +0200\\n\\n nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\\n \\n This only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects' root_dir str '/home/runner/work/spoofax-pie/spoofax-pie' Page Attributes \u00b6 Provided by MkDocs. These attributes change for every page (the attributes shown are for this page). e.g. {{ page.title }} See also the MkDocs documentation on the page object . Variable Type Content file File page [ Page ], src_uri = 'info.md', abs_src_path = '/home/runner/work/spoofax-pie/spoofax-pie/docs/info.md', name = 'info', dest_uri = 'info/index.html', abs_dest_path = '/home/runner/work/spoofax-pie/spoofax-pie/site/info/index.html', url = 'info/' title str 'Info' parent NoneType None children NoneType None previous_page NoneType None next_page NoneType None _Page__active bool False update_date str '2024-07-29' canonical_url NoneType None abs_url NoneType None edit_url str 'https://github.com/metaborg/spoofax-pie/edit/master/docs/info.md' markdown str '{{ macros_info() }}\\n' content NoneType None toc list [] meta dict To have all titles of all pages, use: {% for page in navigation.pages %} - {{ page.title }} {% endfor %} Plugin Filters \u00b6 These filters are provided as a standard by the macros plugin. Variable Type Content pretty function ( var_list, rows, header, e ) Default mkdocs_macro Prettify a dictionary or object (used for environment documentation, or debugging). Builtin Jinja2 Filters \u00b6 These filters are provided by Jinja2 as a standard. See also the Jinja2 documentation on builtin filters ). Variable Type Content abs builtin_function_or_method Return the absolute value of the argument. attr function ( environment, obj, name, value ) Get an attribute of an object. foo|attr(\"bar\") works like foo.bar just that always an attribute is returned and items are not looked up. batch function ( value, linecount, fill_with, tmp, item ) A filter that batches items. It works pretty much like slice just the other way round. It returns a list of lists with the given number of items. If you provide a second parameter this is used to fill up missing items. See this example. capitalize function ( s ) Capitalize a value. The first character will be uppercase, all others lowercase. center function ( value, width ) Centers the value in a field of a given width. count builtin_function_or_method Return the number of items in a container. d function ( value, default_value, boolean ) If the value is undefined it will return the passed default value, otherwise the value of the variable. default function ( value, default_value, boolean ) If the value is undefined it will return the passed default value, otherwise the value of the variable. dictsort function ( value, case_sensitive, by, reverse, sort_func ) Sort a dict and yield (key, value) pairs. Python dicts may not be in the order you want to display them in, so sort them first. e builtin_function_or_method Replace the characters & , < , > , ' , and \" in the string with HTML-safe sequences. Use this if you need to display text that might contain such characters in HTML. escape builtin_function_or_method Replace the characters & , < , > , ' , and \" in the string with HTML-safe sequences. Use this if you need to display text that might contain such characters in HTML. filesizeformat function ( value, binary, bytes, base, prefixes, i, prefix, unit ) Format the value like a 'human-readable' file size (i.e. 13 kB, 4.1 MB, 102 Bytes, etc). Per default decimal prefixes are used (Mega, Giga, etc.), if the second parameter is set to True the binary prefixes are used (Mebi, Gibi). first function ( args, kwargs, b ) Return the first item of a sequence. float function ( value, default ) Convert the value into a floating point number. If the conversion doesn't work it will return 0.0 . You can override this default using the first parameter. forceescape function ( value ) Enforce HTML escaping. This will probably double escape variables. format function ( value, args, kwargs ) Apply the given values to a printf-style _ format string, like string % values . groupby function ( args, kwargs, b ) Group a sequence of objects by an attribute using Python's :func: itertools.groupby . The attribute can use dot notation for nested access, like \"address.city\" . Unlike Python's groupby , the values are sorted first so only one group is returned for each unique value. indent function ( s, width, first, blank, newline, rv, lines ) Return a copy of the string with each line indented by 4 spaces. The first line and blank lines are not indented by default. int function ( value, default, base ) Convert the value into an integer. If the conversion doesn't work it will return 0 . You can override this default using the first parameter. You can also override the default base (10) in the second parameter, which handles input with prefixes such as 0b, 0o and 0x for bases 2, 8 and 16 respectively. The base is ignored for decimal numbers and non-string values. join function ( args, kwargs, b ) Return a string which is the concatenation of the strings in the sequence. The separator between elements is an empty string per default, you can define it with the optional parameter. last function ( environment, seq ) Return the last item of a sequence. length builtin_function_or_method Return the number of items in a container. list function ( args, kwargs, b ) Convert the value into a list. If it was a string the returned list will be a list of characters. lower function ( s ) Convert a value to lowercase. items function ( value ) Return an iterator over the (key, value) items of a mapping. map function ( args, kwargs, b ) Applies a filter on a sequence of objects or looks up an attribute. This is useful when dealing with lists of objects but you are really only interested in a certain value of it. min function ( environment, value, case_sensitive, attribute ) Return the smallest item from the sequence. max function ( environment, value, case_sensitive, attribute ) Return the largest item from the sequence. pprint function ( value ) Pretty print a variable. Useful for debugging. random function ( context, seq ) Return a random item from the sequence. reject function ( args, kwargs, b ) Filters a sequence of objects by applying a test to each object, and rejecting the objects with the test succeeding. rejectattr function ( args, kwargs, b ) Filters a sequence of objects by applying a test to the specified attribute of each object, and rejecting the objects with the test succeeding. replace function ( eval_ctx, s, old, new, count ) Return a copy of the value with all occurrences of a substring replaced with a new one. The first argument is the substring that should be replaced, the second is the replacement string. If the optional third argument count is given, only the first count occurrences are replaced. reverse function ( value, rv, e ) Reverse the object or return an iterator that iterates over it the other way round. round function ( value, precision, method, func ) Round the number to a given precision. The first parameter specifies the precision (default is 0 ), the second the rounding method. safe function ( value ) Mark the value as safe which means that in an environment with automatic escaping enabled this variable will not be escaped. select function ( args, kwargs, b ) Filters a sequence of objects by applying a test to each object, and only selecting the objects with the test succeeding. selectattr function ( args, kwargs, b ) Filters a sequence of objects by applying a test to the specified attribute of each object, and only selecting the objects with the test succeeding. slice function ( args, kwargs, b ) Slice an iterator and return a list of lists containing those items. Useful if you want to create a div containing three ul tags that represent columns. sort function ( environment, value, reverse, case_sensitive, attribute, key_func ) Sort an iterable using Python's :func: sorted . string builtin_function_or_method Convert an object to a string if it isn't already. This preserves a :class: Markup string rather than converting it back to a basic string, so it will still be marked as safe and won't be escaped again. striptags function ( value ) Strip SGML/XML tags and replace adjacent whitespace by one space. sum function ( args, kwargs, b ) Returns the sum of a sequence of numbers plus the value of parameter 'start' (which defaults to 0). When the sequence is empty it returns start. title function ( s ) Return a titlecased version of the value. I.e. words will start with uppercase letters, all remaining characters are lowercase. trim function ( value, chars ) Strip leading and trailing characters, by default whitespace. truncate function ( env, s, length, killwords, end, leeway, result ) Return a truncated copy of the string. The length is specified with the first parameter which defaults to 255 . If the second parameter is true the filter will cut the text at length. Otherwise it will discard the last word. If the text was in fact truncated it will append an ellipsis sign ( \"...\" ). If you want a different ellipsis sign than \"...\" you can specify it using the third parameter. Strings that only exceed the length by the tolerance margin given in the fourth parameter will not be truncated. unique function ( environment, value, case_sensitive, attribute, getter, seen, item, key ) Returns a list of unique items from the given iterable. upper function ( s ) Convert a value to uppercase. urlencode function ( value, items ) Quote data for use in a URL path or query using UTF-8. urlize function ( eval_ctx, value, trim_url_limit, nofollow, target, rel, extra_schemes, policies, rel_parts, scheme, rv ) Convert URLs in text into clickable links. wordcount function ( s ) Count the words in that string. wordwrap function ( environment, s, width, break_long_words, wrapstring, break_on_hyphens ) Wrap a string to the given width. Existing newlines are treated as paragraphs to be wrapped separately. xmlattr function ( eval_ctx, d, autospace, items, key, value, rv ) Create an SGML/XML attribute string based on the items in a dict. tojson function ( eval_ctx, value, indent, policies, dumps, kwargs ) Serialize an object to a string of JSON, and mark it safe to render in HTML. This filter is only for use in HTML documents.","title":"Info"},{"location":"info/#macros-plugin-environment","text":"","title":"Macros Plugin Environment"},{"location":"info/#general-list","text":"All available variables and filters within the macros plugin: Variable Type Content extra dict config MkDocsConfig {'config_file_path': '/home/runner/work/spoofax-pie/spoofax-pie/mkdocs.yml', 'site_name': 'Spoofax 3', 'nav': [{'Home': 'index.md'}, {'Tutorials': ['tutorial/install.md', 'tutorial/create_language_project.md', 'tutorial/change_static_semantics.md', 'tutorial/add_transformation.md', 'tutorial/pcf_tutorial.md']}, {'How-to Guides': ['guide/ask_for_help.md', 'guide/report_a_bug.md', {'Eclipse LWB': ['guide/eclipse_lwb/import.md', 'guide/eclipse_lwb/update.md', 'guide/eclipse_lwb/troubleshooting.md']}, {'Static Semantics': ['guide/static-semantics/code-completion.md']}, {'Development': ['guide/development/debugging-in-intellij.md', 'guide/development/troubleshooting.md']}]}, {'Reference': ['reference/configuration.md', 'reference/eclipse-lwb/eclipse-project-files.md']}, {'Background': ['background/documentation.md', 'background/motivation.md', 'background/key_ideas.md', 'background/status.md']}, {'Releases': ['release/download.md']}], 'pages': None, 'site_url': None, 'site_description': 'Spoofax 3 documentation website', 'site_author': None, 'theme': Theme(name='material', dirs=['/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/material', '/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''), language='en', direction=None, features=['navigation.expand', 'navigation.instant', 'navigation.tabs', 'navigation.tabs.sticky', 'navigation.sections', 'navigation.indexes', 'navigation.top'], palette={'primary': 'indigo', 'accent': 'indigo'}, font={'code': 'JetBrains Mono'}, icon=None, favicon='assets/spoofax.png', include_search_page=False, search_index_only=True, logo='assets/spoofax.svg'), 'docs_dir': '/home/runner/work/spoofax-pie/spoofax-pie/docs', 'site_dir': '/home/runner/work/spoofax-pie/spoofax-pie/site', 'copyright': None, 'google_analytics': None, 'dev_addr': _IpAddressValue(host='127.0.0.1', port=8000), 'use_directory_urls': True, 'repo_url': 'https://github.com/metaborg/spoofax-pie', 'repo_name': 'metaborg/spoofax-pie', 'edit_uri_template': None, 'edit_uri': 'edit/master/docs/', 'extra_css': ['extra.css'], 'extra_javascript': [], 'extra_templates': [], 'markdown_extensions': ['toc', 'tables', 'fenced_code', 'abbr', 'admonition', 'attr_list', 'codehilite', 'def_list', 'pymdownx.betterem', 'pymdownx.caret', 'pymdownx.details', 'pymdownx.emoji', 'pymdownx.inlinehilite', 'pymdownx.keys', 'pymdownx.magiclink', 'pymdownx.mark', 'pymdownx.saneheaders', 'pymdownx.smartsymbols', 'pymdownx.snippets', 'pymdownx.superfences', 'pymdownx.tabbed', 'pymdownx.tasklist', 'pymdownx.tilde'], 'mdx_configs': {'codehilite': {'guess_lang': False}, 'toc': {'permalink': True}, 'pymdownx.betterem': {'smart_enable': 'all'}, 'pymdownx.emoji': {'emoji_index': , 'emoji_generator': }, 'pymdownx.tasklist': {'custom_checkbox': True}}, 'strict': False, 'remote_branch': 'gh-pages', 'remote_name': 'origin', 'extra': {}, 'plugins': PluginCollection([('search', ), ('macros', ), ('git-revision-date', )]), 'hooks': {}, 'watch': []} environment dict system = 'Linux', system_version = '6.5.0-1024-azure', python_version = '3.8.18', mkdocs_version = '1.4.0', macros_plugin_version = '0.7.0', jinja2_version = '3.1.4' plugin LegacyConfig {'module_name': 'docs/macro', 'modules': [], 'include_dir': 'docs/include', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '', 'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False} git dict status = True, date [ datetime ], short_commit = 'aec1bcb', commit = 'aec1bcbef07cce68c7411506b99132762273a448', tag = '', author = 'Jeff Smits', author_email = 'j.smits-1@tudelft.nl', committer = 'Jeff Smits', committer_email = 'j.smits-1@tudelft.nl', date_ISO = 'Mon Jul 29 16:25:35 2024 +0200', message = 'nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\\n\\nThis only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects', raw = 'commit aec1bcbef07cce68c7411506b99132762273a448\\nAuthor: Jeff Smits \\nDate: Mon Jul 29 16:25:32 2024 +0200\\n\\n nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\\n \\n This only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects', root_dir = '/home/runner/work/spoofax-pie/spoofax-pie' os dict windows = ':fontawesome-brands-windows: Windows', linux = ':fontawesome-brands-linux: Linux', macos = ':fontawesome-brands-apple: macOS' release dict 0.22.0 [ dict ], 0.20.0 [ dict ], 0.19.8 [ dict ], 0.19.7 [ dict ], 0.19.4 [ dict ], 0.19.3 [ dict ], 0.19.2 [ dict ], 0.19.1 [ dict ], 0.19.0 [ dict ], 0.18.0 [ dict ], 0.17.0 [ dict ], 0.16.17 [ dict ], 0.16.16 [ dict ], 0.16.15 [ dict ], 0.16.14 [ dict ], 0.16.13 [ dict ], 0.16.12 [ dict ], 0.16.11 [ dict ], 0.16.10 [ dict ], 0.16.9 [ dict ], 0.16.8 [ dict ], 0.16.7 [ dict ], 0.16.6 [ dict ], 0.16.5 [ dict ], 0.16.4 [ dict ], 0.16.3 [ dict ], 0.16.2 [ dict ], 0.16.1 [ dict ], 0.16.0 [ dict ], 0.15.3 [ dict ], 0.15.2 [ dict ], 0.15.1 [ dict ], 0.15.0 [ dict ], 0.14.2 [ dict ], 0.14.1 [ dict ], 0.14.0 [ dict ], 0.13.0 [ dict ], 0.12.1 [ dict ], 0.12.0 [ dict ], 0.11.13 [ dict ], 0.11.12 [ dict ], 0.11.11 [ dict ], 0.11.10 [ dict ], 0.11.9 [ dict ], 0.11.8 [ dict ], 0.11.7 [ dict ], 0.11.6 [ dict ], 0.11.5 [ dict ], 0.11.4 [ dict ], 0.11.3 [ dict ], 0.11.2 [ dict ], 0.11.1 [ dict ], 0.11.0 [ dict ], 0.10.0 [ dict ], 0.9.0 [ dict ], 0.8.0 [ dict ], rel [ dict ], dev [ dict ] macros SuperDict context [ function ], macros_info [ function ], now [ function ], fix_url [ function ] filters dict pretty [ function ] filters_builtin dict abs [ builtin_function_or_method ], attr [ function ], batch [ function ], capitalize [ function ], center [ function ], count [ builtin_function_or_method ], d [ function ], default [ function ], dictsort [ function ], e [ builtin_function_or_method ], escape [ builtin_function_or_method ], filesizeformat [ function ], first [ function ], float [ function ], forceescape [ function ], format [ function ], groupby [ function ], indent [ function ], int [ function ], join [ function ], last [ function ], length [ builtin_function_or_method ], list [ function ], lower [ function ], items [ function ], map [ function ], min [ function ], max [ function ], pprint [ function ], random [ function ], reject [ function ], rejectattr [ function ], replace [ function ], reverse [ function ], round [ function ], safe [ function ], select [ function ], selectattr [ function ], slice [ function ], sort [ function ], string [ builtin_function_or_method ], striptags [ function ], sum [ function ], title [ function ], trim [ function ], truncate [ function ], unique [ function ], upper [ function ], urlencode [ function ], urlize [ function ], wordcount [ function ], wordwrap [ function ], xmlattr [ function ], tojson [ function ] navigation Navigation Page(title='Home', url='.') Section(title='Tutorials') Page(title=[blank], url='tutorial/install/') Page(title=[blank], url='tutorial/create_language_project/') Page(title=[blank], url='tutorial/change_static_semantics/') Page(title=[blank], url='tutorial/add_transformation/') Page(title=[blank], url='tutorial/pcf_tutorial/') Section(title='How-to Guides') Page(title=[blank], url='guide/ask_for_help/') Page(title=[blank], url='guide/report_a_bug/') Section(title='Eclipse LWB') Page(title=[blank], url='guide/eclipse_lwb/import/') Page(title=[blank], url='guide/eclipse_lwb/update/') Page(title=[blank], url='guide/eclipse_lwb/troubleshooting/') Section(title='Static Semantics') Page(title=[blank], url='guide/static-semantics/code-completion/') Section(title='Development') Page(title=[blank], url='guide/development/debugging-in-intellij/') Page(title=[blank], url='guide/development/troubleshooting/') Section(title='Reference') Page(title=[blank], url='reference/configuration/') Page(title=[blank], url='reference/eclipse-lwb/eclipse-project-files/') Section(title='Background') Page(title=[blank], url='background/documentation/') Page(title=[blank], url='background/motivation/') Page(title=[blank], url='background/key_ideas/') Page(title=[blank], url='background/status/') Section(title='Releases') Page(title=[blank], url='release/download/') files Files page Page Page(title='Info', url='info/')","title":"General List"},{"location":"info/#config-information","text":"Standard MkDocs configuration information. Do not try to modify. e.g. {{ config.docs_dir }} See also the MkDocs documentation on the config object . Variable Type Content config_file_path str '/home/runner/work/spoofax-pie/spoofax-pie/mkdocs.yml' site_name str 'Spoofax 3' nav list [{'Home': 'index.md'}, {'Tutorials': ['tutorial/install.md', 'tutorial/create_language_project.md', 'tutorial/change_static_semantics.md', 'tutorial/add_transformation.md', 'tutorial/pcf_tutorial.md']}, {'How-to Guides': ['guide/ask_for_help.md', 'guide/report_a_bug.md', {'Eclipse LWB': ['guide/eclipse_lwb/import.md', 'guide/eclipse_lwb/update.md', 'guide/eclipse_lwb/troubleshooting.md']}, {'Static Semantics': ['guide/static-semantics/code-completion.md']}, {'Development': ['guide/development/debugging-in-intellij.md', 'guide/development/troubleshooting.md']}]}, {'Reference': ['reference/configuration.md', 'reference/eclipse-lwb/eclipse-project-files.md']}, {'Background': ['background/documentation.md', 'background/motivation.md', 'background/key_ideas.md', 'background/status.md']}, {'Releases': ['release/download.md']}] pages NoneType None site_url NoneType None site_description str 'Spoofax 3 documentation website' site_author NoneType None theme Theme Theme(name='material', dirs=['/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/material', '/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''), language='en', direction=None, features=['navigation.expand', 'navigation.instant', 'navigation.tabs', 'navigation.tabs.sticky', 'navigation.sections', 'navigation.indexes', 'navigation.top'], palette={'primary': 'indigo', 'accent': 'indigo'}, font={'code': 'JetBrains Mono'}, icon=None, favicon='assets/spoofax.png', include_search_page=False, search_index_only=True, logo='assets/spoofax.svg') docs_dir str '/home/runner/work/spoofax-pie/spoofax-pie/docs' site_dir str '/home/runner/work/spoofax-pie/spoofax-pie/site' copyright NoneType None google_analytics NoneType None dev_addr _IpAddressValue _IpAddressValue(host='127.0.0.1', port=8000) use_directory_urls bool True repo_url str 'https://github.com/metaborg/spoofax-pie' repo_name str 'metaborg/spoofax-pie' edit_uri_template NoneType None edit_uri str 'edit/master/docs/' extra_css list ['extra.css'] extra_javascript list [] extra_templates list [] markdown_extensions list ['toc', 'tables', 'fenced_code', 'abbr', 'admonition', 'attr_list', 'codehilite', 'def_list', 'pymdownx.betterem', 'pymdownx.caret', 'pymdownx.details', 'pymdownx.emoji', 'pymdownx.inlinehilite', 'pymdownx.keys', 'pymdownx.magiclink', 'pymdownx.mark', 'pymdownx.saneheaders', 'pymdownx.smartsymbols', 'pymdownx.snippets', 'pymdownx.superfences', 'pymdownx.tabbed', 'pymdownx.tasklist', 'pymdownx.tilde'] mdx_configs dict codehilite [ dict ], toc [ dict ], pymdownx.betterem [ dict ], pymdownx.emoji [ dict ], pymdownx.tasklist [ dict ] strict bool False remote_branch str 'gh-pages' remote_name str 'origin' extra LegacyConfig {} plugins PluginCollection search [ SearchPlugin ], macros [ MacrosPlugin ], git-revision-date [ GitRevisionDatePlugin ] hooks dict watch list []","title":"Config Information"},{"location":"info/#macros","text":"These macros have been defined programmatically for this environment (module or pluglets). Variable Type Content context function ( obj, e ) Default mkdocs_macro List the defined variables macros_info function ( ) Test/debug function: list useful documentation on the mkdocs_macro environment. now function ( ) Get the current time (returns a datetime object). Used alone, it provides a timestamp. To get the year use now().year , for the month number now().month , etc. fix_url function ( url, r ) If url is relative, fix it so that it points to the docs diretory. This is necessary because relative links in markdown must be adapted in html ('img/foo.png' => '../img/img.png').","title":"Macros"},{"location":"info/#git-information","text":"Information available on the last commit and the git repository containing the documentation project: e.g. {{ git.message }} Variable Type Content status bool True date datetime datetime.datetime(2024, 7, 29, 16, 25, 35, tzinfo=tzoffset(None, 7200)) short_commit str 'aec1bcb' commit str 'aec1bcbef07cce68c7411506b99132762273a448' tag str '' author str 'Jeff Smits' author_email str 'j.smits-1@tudelft.nl' committer str 'Jeff Smits' committer_email str 'j.smits-1@tudelft.nl' date_ISO str 'Mon Jul 29 16:25:35 2024 +0200' message str 'nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\\n\\nThis only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects' raw str 'commit aec1bcbef07cce68c7411506b99132762273a448\\nAuthor: Jeff Smits \\nDate: Mon Jul 29 16:25:32 2024 +0200\\n\\n nabl2.terms should be api to pass it along the deps to e.g. statix.intellij\\n \\n This only took about a week to debug and find... statix.eclipse would have the same issue of not finding nabl2.terms but because the BundlePlugin extends the compile classpath with the bundle class paths it still ends up there. This problem is found when we use a newer version of dagger and/or newer versions of the Spoofax 2 projects' root_dir str '/home/runner/work/spoofax-pie/spoofax-pie'","title":"Git Information"},{"location":"info/#page-attributes","text":"Provided by MkDocs. These attributes change for every page (the attributes shown are for this page). e.g. {{ page.title }} See also the MkDocs documentation on the page object . Variable Type Content file File page [ Page ], src_uri = 'info.md', abs_src_path = '/home/runner/work/spoofax-pie/spoofax-pie/docs/info.md', name = 'info', dest_uri = 'info/index.html', abs_dest_path = '/home/runner/work/spoofax-pie/spoofax-pie/site/info/index.html', url = 'info/' title str 'Info' parent NoneType None children NoneType None previous_page NoneType None next_page NoneType None _Page__active bool False update_date str '2024-07-29' canonical_url NoneType None abs_url NoneType None edit_url str 'https://github.com/metaborg/spoofax-pie/edit/master/docs/info.md' markdown str '{{ macros_info() }}\\n' content NoneType None toc list [] meta dict To have all titles of all pages, use: {% for page in navigation.pages %} - {{ page.title }} {% endfor %}","title":"Page Attributes"},{"location":"info/#plugin-filters","text":"These filters are provided as a standard by the macros plugin. Variable Type Content pretty function ( var_list, rows, header, e ) Default mkdocs_macro Prettify a dictionary or object (used for environment documentation, or debugging).","title":"Plugin Filters"},{"location":"info/#builtin-jinja2-filters","text":"These filters are provided by Jinja2 as a standard. See also the Jinja2 documentation on builtin filters ). Variable Type Content abs builtin_function_or_method Return the absolute value of the argument. attr function ( environment, obj, name, value ) Get an attribute of an object. foo|attr(\"bar\") works like foo.bar just that always an attribute is returned and items are not looked up. batch function ( value, linecount, fill_with, tmp, item ) A filter that batches items. It works pretty much like slice just the other way round. It returns a list of lists with the given number of items. If you provide a second parameter this is used to fill up missing items. See this example. capitalize function ( s ) Capitalize a value. The first character will be uppercase, all others lowercase. center function ( value, width ) Centers the value in a field of a given width. count builtin_function_or_method Return the number of items in a container. d function ( value, default_value, boolean ) If the value is undefined it will return the passed default value, otherwise the value of the variable. default function ( value, default_value, boolean ) If the value is undefined it will return the passed default value, otherwise the value of the variable. dictsort function ( value, case_sensitive, by, reverse, sort_func ) Sort a dict and yield (key, value) pairs. Python dicts may not be in the order you want to display them in, so sort them first. e builtin_function_or_method Replace the characters & , < , > , ' , and \" in the string with HTML-safe sequences. Use this if you need to display text that might contain such characters in HTML. escape builtin_function_or_method Replace the characters & , < , > , ' , and \" in the string with HTML-safe sequences. Use this if you need to display text that might contain such characters in HTML. filesizeformat function ( value, binary, bytes, base, prefixes, i, prefix, unit ) Format the value like a 'human-readable' file size (i.e. 13 kB, 4.1 MB, 102 Bytes, etc). Per default decimal prefixes are used (Mega, Giga, etc.), if the second parameter is set to True the binary prefixes are used (Mebi, Gibi). first function ( args, kwargs, b ) Return the first item of a sequence. float function ( value, default ) Convert the value into a floating point number. If the conversion doesn't work it will return 0.0 . You can override this default using the first parameter. forceescape function ( value ) Enforce HTML escaping. This will probably double escape variables. format function ( value, args, kwargs ) Apply the given values to a printf-style _ format string, like string % values . groupby function ( args, kwargs, b ) Group a sequence of objects by an attribute using Python's :func: itertools.groupby . The attribute can use dot notation for nested access, like \"address.city\" . Unlike Python's groupby , the values are sorted first so only one group is returned for each unique value. indent function ( s, width, first, blank, newline, rv, lines ) Return a copy of the string with each line indented by 4 spaces. The first line and blank lines are not indented by default. int function ( value, default, base ) Convert the value into an integer. If the conversion doesn't work it will return 0 . You can override this default using the first parameter. You can also override the default base (10) in the second parameter, which handles input with prefixes such as 0b, 0o and 0x for bases 2, 8 and 16 respectively. The base is ignored for decimal numbers and non-string values. join function ( args, kwargs, b ) Return a string which is the concatenation of the strings in the sequence. The separator between elements is an empty string per default, you can define it with the optional parameter. last function ( environment, seq ) Return the last item of a sequence. length builtin_function_or_method Return the number of items in a container. list function ( args, kwargs, b ) Convert the value into a list. If it was a string the returned list will be a list of characters. lower function ( s ) Convert a value to lowercase. items function ( value ) Return an iterator over the (key, value) items of a mapping. map function ( args, kwargs, b ) Applies a filter on a sequence of objects or looks up an attribute. This is useful when dealing with lists of objects but you are really only interested in a certain value of it. min function ( environment, value, case_sensitive, attribute ) Return the smallest item from the sequence. max function ( environment, value, case_sensitive, attribute ) Return the largest item from the sequence. pprint function ( value ) Pretty print a variable. Useful for debugging. random function ( context, seq ) Return a random item from the sequence. reject function ( args, kwargs, b ) Filters a sequence of objects by applying a test to each object, and rejecting the objects with the test succeeding. rejectattr function ( args, kwargs, b ) Filters a sequence of objects by applying a test to the specified attribute of each object, and rejecting the objects with the test succeeding. replace function ( eval_ctx, s, old, new, count ) Return a copy of the value with all occurrences of a substring replaced with a new one. The first argument is the substring that should be replaced, the second is the replacement string. If the optional third argument count is given, only the first count occurrences are replaced. reverse function ( value, rv, e ) Reverse the object or return an iterator that iterates over it the other way round. round function ( value, precision, method, func ) Round the number to a given precision. The first parameter specifies the precision (default is 0 ), the second the rounding method. safe function ( value ) Mark the value as safe which means that in an environment with automatic escaping enabled this variable will not be escaped. select function ( args, kwargs, b ) Filters a sequence of objects by applying a test to each object, and only selecting the objects with the test succeeding. selectattr function ( args, kwargs, b ) Filters a sequence of objects by applying a test to the specified attribute of each object, and only selecting the objects with the test succeeding. slice function ( args, kwargs, b ) Slice an iterator and return a list of lists containing those items. Useful if you want to create a div containing three ul tags that represent columns. sort function ( environment, value, reverse, case_sensitive, attribute, key_func ) Sort an iterable using Python's :func: sorted . string builtin_function_or_method Convert an object to a string if it isn't already. This preserves a :class: Markup string rather than converting it back to a basic string, so it will still be marked as safe and won't be escaped again. striptags function ( value ) Strip SGML/XML tags and replace adjacent whitespace by one space. sum function ( args, kwargs, b ) Returns the sum of a sequence of numbers plus the value of parameter 'start' (which defaults to 0). When the sequence is empty it returns start. title function ( s ) Return a titlecased version of the value. I.e. words will start with uppercase letters, all remaining characters are lowercase. trim function ( value, chars ) Strip leading and trailing characters, by default whitespace. truncate function ( env, s, length, killwords, end, leeway, result ) Return a truncated copy of the string. The length is specified with the first parameter which defaults to 255 . If the second parameter is true the filter will cut the text at length. Otherwise it will discard the last word. If the text was in fact truncated it will append an ellipsis sign ( \"...\" ). If you want a different ellipsis sign than \"...\" you can specify it using the third parameter. Strings that only exceed the length by the tolerance margin given in the fourth parameter will not be truncated. unique function ( environment, value, case_sensitive, attribute, getter, seen, item, key ) Returns a list of unique items from the given iterable. upper function ( s ) Convert a value to uppercase. urlencode function ( value, items ) Quote data for use in a URL path or query using UTF-8. urlize function ( eval_ctx, value, trim_url_limit, nofollow, target, rel, extra_schemes, policies, rel_parts, scheme, rv ) Convert URLs in text into clickable links. wordcount function ( s ) Count the words in that string. wordwrap function ( environment, s, width, break_long_words, wrapstring, break_on_hyphens ) Wrap a string to the given width. Existing newlines are treated as paragraphs to be wrapped separately. xmlattr function ( eval_ctx, d, autospace, items, key, value, rv ) Create an SGML/XML attribute string based on the items in a dict. tojson function ( eval_ctx, value, indent, policies, dumps, kwargs ) Serialize an object to a string of JSON, and mark it safe to render in HTML. This filter is only for use in HTML documents.","title":"Builtin Jinja2 Filters"},{"location":"_include/_all/","text":"","title":" all"},{"location":"_include/abbreviation/","text":"","title":"Abbreviation"},{"location":"background/documentation/","text":"Documentation \u00b6 In this section, we explain the documentation technology and how it is structured. Technology \u00b6 This documentation is developed with MkDocs , a fast and simple static site generated that's geared towards building project documentation from Markdown files. MkDocs uses Python-Markdown to process Markdown files, along with PyMdown Extensions . We use the Material for MkDocs theme which provides a clean look, easy customization, and many features for technical documentation. We use the following MkDocs plugins: mkdocs-macros-plugin to enable the use of variables, macros, and filters in Markdown files. mkdocs-git-revision-date-plugin to add a changed date to the footer based on the last time the file was changed in the Git repository. The documentation is automatically built and published on a commit to the master branch of this repository using the GitHub actions workflow at .github/workflows/documentation.yml . Structure \u00b6 The structure of this documentation follows The documentation system where documentation is split into four categories: Tutorials : oriented to learning , enabling newcomers to get started through a lesson , analogous to teaching a child how to cook . How-to guides : oriented to a particular goal , showing how to solve a specific problem through a series of steps , analogous to a recipe in a cookery book . Reference : oriented to information , describing the machinery through dry description , analogous to an encyclopaedia article . Explanation : oriented to understanding , explaining through discursive explanation , analogous to an article on culinary social history .","title":"Documentation"},{"location":"background/documentation/#documentation","text":"In this section, we explain the documentation technology and how it is structured.","title":"Documentation"},{"location":"background/documentation/#technology","text":"This documentation is developed with MkDocs , a fast and simple static site generated that's geared towards building project documentation from Markdown files. MkDocs uses Python-Markdown to process Markdown files, along with PyMdown Extensions . We use the Material for MkDocs theme which provides a clean look, easy customization, and many features for technical documentation. We use the following MkDocs plugins: mkdocs-macros-plugin to enable the use of variables, macros, and filters in Markdown files. mkdocs-git-revision-date-plugin to add a changed date to the footer based on the last time the file was changed in the Git repository. The documentation is automatically built and published on a commit to the master branch of this repository using the GitHub actions workflow at .github/workflows/documentation.yml .","title":"Technology"},{"location":"background/documentation/#structure","text":"The structure of this documentation follows The documentation system where documentation is split into four categories: Tutorials : oriented to learning , enabling newcomers to get started through a lesson , analogous to teaching a child how to cook . How-to guides : oriented to a particular goal , showing how to solve a specific problem through a series of steps , analogous to a recipe in a cookery book . Reference : oriented to information , describing the machinery through dry description , analogous to an encyclopaedia article . Explanation : oriented to understanding , explaining through discursive explanation , analogous to an article on culinary social history .","title":"Structure"},{"location":"background/key_ideas/","text":"Key ideas \u00b6 To solve the problems highlighted in the motivation section, we intend to employ the following key ideas in Spoofax 3. To reduce coupling, Spoofax 3's \"Spoofax Core\" does not depend on any meta-components. Instead, a language implementation depends directly on the meta-components that it requires. For example, the Tiger language implementation depends directly on the JSGLR2 parser, the NaBL2 constraint solver, and the Stratego runtime. To make language pipelines flexible, modular, and incremental, we use an incremental, modular, and expressive build system as the basis for creating pipelines: PIE . Language processing steps such as parsing, styling text, analyzing, checking (to provide inline error messages), running (parts of) a compiler, etc. become PIE task definitions. Tasks, which are instances of these task definitions, can depend on each other, and depend on resources such as files. The PIE runtime efficiently and incrementally executes tasks. Furthermore, task definitions can be shared and used by other language implementations, making language implementations modular. To reduce the tedium of dynamic language loading, we instead choose to do static language loading as the default. A language implementation is just a JAR file that can be put on the classpath and used as a regular Java library. For example, to use the JSGLR2 parser of the Tiger language, we just depend on the Tiger language implementation as we would depend on a regular Java library, create an instance of the TigerParser class, and then use that to parse a string into an AST. We still want to automatically provide integrations with the command-line, build systems such as Gradle, and IDEs such as Eclipse and IntelliJ. Therefore, every language implementation must implement the LanguageInstance interface. Spoofax 3 then provides libraries which take a LanguageInstance object, and integrate it with a platform. For example, spoofax.cli takes a LanguageInstance object and provides a command-line application, and spoofax.eclipse does the same for an Eclipse plugin. Because language implementations are just regular Java libraries, they now require some Java boilerplate. However, we do not want language developers to write this Java boilerplate for standard cases. Therefore, we employ a Spoofax 3 compiler that generates this Java boilerplate. If the language developer is not happy with the implementation, or wants to customize parts, they can manually implement or extend Java classes where needed. It is also possible to not use the Spoofax 3 compiler at all, and manually implement all parts. To enable quick language prototyping, we still support dynamic language loading in environments that support them (e.g., Eclipse and IntelliJ), by dynamically loading the language implementation JAR when changed. For example, when prototyping the Tiger language in Eclipse, if the syntax definition is changed we run the Spoofax 3 compiler to (incrementally) create a new parse table and Java classes, and dynamically (re)load the JAR. To improve the user experience, we use a configuration DSL to configure language specifications and implementations. Thereby configuration is centralized, has domain-specific checking, and editor services such as inline errors and code completion. We also allow changing of defaults (conventions), and persist them to enable renaming. To improve error traceability, errors are reported inline where possible. Errors are traced through PIE pipelines and support origin tracking to easily support error traceability and inline errors for all language implementations. TODO: better builders: non-Stratego commands incremental commands separate commands from how they are executed support command parameters/arguments continuous execution TODO: modular and incremental development of Spoofax 3 itself with Gradle","title":"Key ideas"},{"location":"background/key_ideas/#key-ideas","text":"To solve the problems highlighted in the motivation section, we intend to employ the following key ideas in Spoofax 3. To reduce coupling, Spoofax 3's \"Spoofax Core\" does not depend on any meta-components. Instead, a language implementation depends directly on the meta-components that it requires. For example, the Tiger language implementation depends directly on the JSGLR2 parser, the NaBL2 constraint solver, and the Stratego runtime. To make language pipelines flexible, modular, and incremental, we use an incremental, modular, and expressive build system as the basis for creating pipelines: PIE . Language processing steps such as parsing, styling text, analyzing, checking (to provide inline error messages), running (parts of) a compiler, etc. become PIE task definitions. Tasks, which are instances of these task definitions, can depend on each other, and depend on resources such as files. The PIE runtime efficiently and incrementally executes tasks. Furthermore, task definitions can be shared and used by other language implementations, making language implementations modular. To reduce the tedium of dynamic language loading, we instead choose to do static language loading as the default. A language implementation is just a JAR file that can be put on the classpath and used as a regular Java library. For example, to use the JSGLR2 parser of the Tiger language, we just depend on the Tiger language implementation as we would depend on a regular Java library, create an instance of the TigerParser class, and then use that to parse a string into an AST. We still want to automatically provide integrations with the command-line, build systems such as Gradle, and IDEs such as Eclipse and IntelliJ. Therefore, every language implementation must implement the LanguageInstance interface. Spoofax 3 then provides libraries which take a LanguageInstance object, and integrate it with a platform. For example, spoofax.cli takes a LanguageInstance object and provides a command-line application, and spoofax.eclipse does the same for an Eclipse plugin. Because language implementations are just regular Java libraries, they now require some Java boilerplate. However, we do not want language developers to write this Java boilerplate for standard cases. Therefore, we employ a Spoofax 3 compiler that generates this Java boilerplate. If the language developer is not happy with the implementation, or wants to customize parts, they can manually implement or extend Java classes where needed. It is also possible to not use the Spoofax 3 compiler at all, and manually implement all parts. To enable quick language prototyping, we still support dynamic language loading in environments that support them (e.g., Eclipse and IntelliJ), by dynamically loading the language implementation JAR when changed. For example, when prototyping the Tiger language in Eclipse, if the syntax definition is changed we run the Spoofax 3 compiler to (incrementally) create a new parse table and Java classes, and dynamically (re)load the JAR. To improve the user experience, we use a configuration DSL to configure language specifications and implementations. Thereby configuration is centralized, has domain-specific checking, and editor services such as inline errors and code completion. We also allow changing of defaults (conventions), and persist them to enable renaming. To improve error traceability, errors are reported inline where possible. Errors are traced through PIE pipelines and support origin tracking to easily support error traceability and inline errors for all language implementations. TODO: better builders: non-Stratego commands incremental commands separate commands from how they are executed support command parameters/arguments continuous execution TODO: modular and incremental development of Spoofax 3 itself with Gradle","title":"Key ideas"},{"location":"background/motivation/","text":"Motivation \u00b6 In this section we discuss the motivations for developing Spoofax 3. Architecture \u00b6 The main motivation for developing Spoofax 3 is the monolithic, inflexible, and non-incremental architecture of Spoofax 2: It has an inflexible fixed-function pipeline , where every file of your language is parsed, analyzed, and transformed. This works fine, and can even be incremental when the files of your language can be separately compiled. However, this is often not the case. Languages should be able to define their own incremental pipelines with minimal effort. Those pipelines should be modular and flexible, enabling usage in a wide range applications such as command-line interfaces, build systems, code editors, and IDEs. It is monolithic for language users (i.e., the users of your programming language that you have developed with Spoofax), as every language developed with Spoofax 2 depends on Spoofax Core, which in turn depends on all meta-components: JSGLR1 and 2, NaBL+TS index and task engine, NaBL2 & Statix solver, dynsem interpreter, Stratego runtime, config parsing libraries, etc. A language should only require the meta-components that it uses. It is monolithic for meta-component developers (e.g., the developers of the language workbench, or researchers experimenting with new meta-tools or meta-languages). New meta-components need to be tightly integrated into Spoofax Core, requiring time-consuming changes and introducing increased coupling. We should develop meta-components in separation, and loosely couple/integrate them (as much as possible). The build of Spoofax 2 itself is monolithic and non-incremental, as all its components are compiled in one huge non-incremental build, massively increasing iteration time during development. The build must be incremental, and components should be separated where possible to reduce coupling, decreasing iteration times. Language loading \u00b6 Furthermore, Spoofax 2 only support dynamic loading of languages , where a language can be (re)loaded into the running environment. This is very useful during language development, as it enables fast prototyping. However, when we want to statically load the language, we still need to perform the dynamic loading ritual: somehow include the language archive in your application, and then load it at runtime. This hurts startup time, is not supported on some platforms (e.g., Graal native image), and is tedious. We should support both static and dynamic loading (where possible). Error tracing \u00b6 Some errors are not being traced back to their source. For example, many errors in configuration only show up during build time (in the console, which may be hidden) and are not traced back to the configuration file. This confuses users, and may get stuck on simple things, which then require help from us. Errors, warnings, and informational messages should be traced back to their source, and shown inline at the source in IDE environments, or shown as a list of messages with origin information on the command-line. When there are errors, execution should continue in certain instances (e.g., parse error should recover and try to do analysis), but should not in others (e.g., error from static analysis should prevent execution since it could crash). Configuration \u00b6 Another issue is the scattered configuration in language specifications, which is spread over many different places: metaborg.yaml editor/*.esv dynsem.properties In meta-languages files. For example, template options in SDF3. pom.xml .mvn/extensions.xml Finding the right configuration option in these files, and keeping them in sync, is tedious. Furthermore, while most configuration is documented on our website, looking that up still causes a cognitive gap. We should consolidate configuration that belongs together, and not have any duplicate configuration that needs to be kept in sync. Configuration should be supported with editor services such as inline errors and code completion, if possible. Moreover, some parts of a language specification are configured by convention, and these conventions cannot be changed. For example, the main SDF3 file is always assumed to be syntax/.sdf3 . When the language name is changed, but we forget to change the name of this main file, no parse table is built. Configuration conventions should be changeable, and defaults should be persisted to ensure that renamings do not break things. Summary of Problems \u00b6 To summarize, Spoofax 2 suffers from the following problems that form the motivation for Spoofax 3: Monolithic, inflexible, and non-incremental architecture causing: Inflexible and slow language processing due to non-incremental fixed-function pipeline Coupling in Spoofax Core: every language depends on Spoofax Core, and Spoofax Core depends on all meta-components Slow iteration times when developing Spoofax 2 due to its monolithic and non-incremental build Tedious to use languages due to dynamic language loading Confusing (end-)user experience due to: Bad error traceability Scattered configuration Non-incremental configuration (restarts required to update configuration)","title":"Motivation"},{"location":"background/motivation/#motivation","text":"In this section we discuss the motivations for developing Spoofax 3.","title":"Motivation"},{"location":"background/motivation/#architecture","text":"The main motivation for developing Spoofax 3 is the monolithic, inflexible, and non-incremental architecture of Spoofax 2: It has an inflexible fixed-function pipeline , where every file of your language is parsed, analyzed, and transformed. This works fine, and can even be incremental when the files of your language can be separately compiled. However, this is often not the case. Languages should be able to define their own incremental pipelines with minimal effort. Those pipelines should be modular and flexible, enabling usage in a wide range applications such as command-line interfaces, build systems, code editors, and IDEs. It is monolithic for language users (i.e., the users of your programming language that you have developed with Spoofax), as every language developed with Spoofax 2 depends on Spoofax Core, which in turn depends on all meta-components: JSGLR1 and 2, NaBL+TS index and task engine, NaBL2 & Statix solver, dynsem interpreter, Stratego runtime, config parsing libraries, etc. A language should only require the meta-components that it uses. It is monolithic for meta-component developers (e.g., the developers of the language workbench, or researchers experimenting with new meta-tools or meta-languages). New meta-components need to be tightly integrated into Spoofax Core, requiring time-consuming changes and introducing increased coupling. We should develop meta-components in separation, and loosely couple/integrate them (as much as possible). The build of Spoofax 2 itself is monolithic and non-incremental, as all its components are compiled in one huge non-incremental build, massively increasing iteration time during development. The build must be incremental, and components should be separated where possible to reduce coupling, decreasing iteration times.","title":"Architecture"},{"location":"background/motivation/#language-loading","text":"Furthermore, Spoofax 2 only support dynamic loading of languages , where a language can be (re)loaded into the running environment. This is very useful during language development, as it enables fast prototyping. However, when we want to statically load the language, we still need to perform the dynamic loading ritual: somehow include the language archive in your application, and then load it at runtime. This hurts startup time, is not supported on some platforms (e.g., Graal native image), and is tedious. We should support both static and dynamic loading (where possible).","title":"Language loading"},{"location":"background/motivation/#error-tracing","text":"Some errors are not being traced back to their source. For example, many errors in configuration only show up during build time (in the console, which may be hidden) and are not traced back to the configuration file. This confuses users, and may get stuck on simple things, which then require help from us. Errors, warnings, and informational messages should be traced back to their source, and shown inline at the source in IDE environments, or shown as a list of messages with origin information on the command-line. When there are errors, execution should continue in certain instances (e.g., parse error should recover and try to do analysis), but should not in others (e.g., error from static analysis should prevent execution since it could crash).","title":"Error tracing"},{"location":"background/motivation/#configuration","text":"Another issue is the scattered configuration in language specifications, which is spread over many different places: metaborg.yaml editor/*.esv dynsem.properties In meta-languages files. For example, template options in SDF3. pom.xml .mvn/extensions.xml Finding the right configuration option in these files, and keeping them in sync, is tedious. Furthermore, while most configuration is documented on our website, looking that up still causes a cognitive gap. We should consolidate configuration that belongs together, and not have any duplicate configuration that needs to be kept in sync. Configuration should be supported with editor services such as inline errors and code completion, if possible. Moreover, some parts of a language specification are configured by convention, and these conventions cannot be changed. For example, the main SDF3 file is always assumed to be syntax/.sdf3 . When the language name is changed, but we forget to change the name of this main file, no parse table is built. Configuration conventions should be changeable, and defaults should be persisted to ensure that renamings do not break things.","title":"Configuration"},{"location":"background/motivation/#summary-of-problems","text":"To summarize, Spoofax 2 suffers from the following problems that form the motivation for Spoofax 3: Monolithic, inflexible, and non-incremental architecture causing: Inflexible and slow language processing due to non-incremental fixed-function pipeline Coupling in Spoofax Core: every language depends on Spoofax Core, and Spoofax Core depends on all meta-components Slow iteration times when developing Spoofax 2 due to its monolithic and non-incremental build Tedious to use languages due to dynamic language loading Confusing (end-)user experience due to: Bad error traceability Scattered configuration Non-incremental configuration (restarts required to update configuration)","title":"Summary of Problems"},{"location":"background/status/","text":"Current Status \u00b6 We have stated our key ideas, but since Spoofax 3 is still under heavy development, they have not all been implemented yet. We now discuss the current status of Spoofax 3 by summarizing the key ideas and whether they has been implemented, along with any comments. Decoupling : Spoofax Core not depend on any meta-components. Language implementations instead depend on the meta-components they require. Flexible, modular and incremental pipelines : Use PIE . Static loading : Use static loading by default, making language implementation plain JAR files, which are easy to use in the Java ecosystem. LanguageInstance interface : Language implementations must implement the LanguageInstance interface, which a platform library uses to integrate a language with the platform. An initial version of the LanguageInstance interface exists, but this interface is not yet stable and will receive many new features. Currently, this interface contains features pertaining both command-line platforms and IDE/code editor platforms. These may be split up in the future. Generate Java boilerplate : Generate the Java boilerplate that Spoofax 3 now requires due to the LanguageInstance interface and language implementations being plain JAR files. Configuration for Spoofax 3 language implementations based on Spoofax 2 language definitions is provided through a Gradle build script, which is verbose. Quick language prototyping : Support dynamic language loading in environments that support this, to enable quick language prototyping. Configuration DSL : Use a configuration DSL to improve the developer/user experience. Error origin tracking : Perform origin tracking and propagation on errors to improve the developer/user experience. Not all PIE tasks trace errors, and some errors do not have location information yet. Commands : More flexible and incremental version of \"builders\" from Spoofax 2. Non-Stratego commands : Commands execute PIE tasks, which execute Java code. Incremental commands : Commands are incremental because they execute PIE tasks. Separate commands from how they are executed : Commands can be bound to IDE/editor menus, command-line commands, or to resource changes. Command parameters/arguments : Commands can specify parameters, which must be provided as arguments when executed. Modular and incremental development : Use Gradle (instead of Maven) to build Spoofax 3, which increases modularity and provides incremental builds for faster iteration times. Certain changes to core components may trigger long rebuilds, as a lot of projects (indirectly) depend on these core components and require recompilation. Certain changes trigger recompilation of Gradle plugins which are required by the rest of the build. This may cause a long configuration phase which is not parallelized. Our Gradle plugins do not support the Gradle build cache yet. Our Gradle plugins do not support the configuration cache yet. Sometimes multiple imports into IntelliJ are required to have it recognize all dependencies. Furthermore, we now discuss the status of features that were not new key ideas. Language builds Meta-language bootstrapping Bootstrapping requires implementation of the meta-languages in Spoofax 3, which we have not done yet. Meta-tools Syntax specification SDF3 Parsing JSGLR1 JSGLR2 Incremental parsing (but incompatible with recovery) Styling specification ESV (syntax-based) Semantic analysis NaBL2 Only supported for Spoofax 2-based language definitions Statix Statix signature generation based on SDF3 specification FlowSpec Stratego 2 Transformation (compilation) Stratego 2 Testing SPT Not all expectations have been ported over yet Editor services Syntax-based styling Inline error/warning/note messages Reference resolution Hover tooltips Code completion Syntactic Semantic (i.e., based on static semantics) Outline Platforms Command-line Eclipse Concurrency/parallelism is mostly ignored. Therefore, things may run concurrently that are not suppose to which cause data races and crashes. Several editor services and other conveniences are still missing or work in progress. IntelliJ A very minimal IntelliJ plugin for your language is provided, currently only supporting syntax highlighting and inline parse errors. Gradle Maven REPL The following features are being prototyped/experimented with Spoofax 3: Multi-lingual semantic analysis with Statix (Aron Zwaan) Semantic code completion based on Statix specification (Daniel Pelsmaeker) The following features will most likely not be supported: Analysis with NaBL/TS","title":"Current Status"},{"location":"background/status/#current-status","text":"We have stated our key ideas, but since Spoofax 3 is still under heavy development, they have not all been implemented yet. We now discuss the current status of Spoofax 3 by summarizing the key ideas and whether they has been implemented, along with any comments. Decoupling : Spoofax Core not depend on any meta-components. Language implementations instead depend on the meta-components they require. Flexible, modular and incremental pipelines : Use PIE . Static loading : Use static loading by default, making language implementation plain JAR files, which are easy to use in the Java ecosystem. LanguageInstance interface : Language implementations must implement the LanguageInstance interface, which a platform library uses to integrate a language with the platform. An initial version of the LanguageInstance interface exists, but this interface is not yet stable and will receive many new features. Currently, this interface contains features pertaining both command-line platforms and IDE/code editor platforms. These may be split up in the future. Generate Java boilerplate : Generate the Java boilerplate that Spoofax 3 now requires due to the LanguageInstance interface and language implementations being plain JAR files. Configuration for Spoofax 3 language implementations based on Spoofax 2 language definitions is provided through a Gradle build script, which is verbose. Quick language prototyping : Support dynamic language loading in environments that support this, to enable quick language prototyping. Configuration DSL : Use a configuration DSL to improve the developer/user experience. Error origin tracking : Perform origin tracking and propagation on errors to improve the developer/user experience. Not all PIE tasks trace errors, and some errors do not have location information yet. Commands : More flexible and incremental version of \"builders\" from Spoofax 2. Non-Stratego commands : Commands execute PIE tasks, which execute Java code. Incremental commands : Commands are incremental because they execute PIE tasks. Separate commands from how they are executed : Commands can be bound to IDE/editor menus, command-line commands, or to resource changes. Command parameters/arguments : Commands can specify parameters, which must be provided as arguments when executed. Modular and incremental development : Use Gradle (instead of Maven) to build Spoofax 3, which increases modularity and provides incremental builds for faster iteration times. Certain changes to core components may trigger long rebuilds, as a lot of projects (indirectly) depend on these core components and require recompilation. Certain changes trigger recompilation of Gradle plugins which are required by the rest of the build. This may cause a long configuration phase which is not parallelized. Our Gradle plugins do not support the Gradle build cache yet. Our Gradle plugins do not support the configuration cache yet. Sometimes multiple imports into IntelliJ are required to have it recognize all dependencies. Furthermore, we now discuss the status of features that were not new key ideas. Language builds Meta-language bootstrapping Bootstrapping requires implementation of the meta-languages in Spoofax 3, which we have not done yet. Meta-tools Syntax specification SDF3 Parsing JSGLR1 JSGLR2 Incremental parsing (but incompatible with recovery) Styling specification ESV (syntax-based) Semantic analysis NaBL2 Only supported for Spoofax 2-based language definitions Statix Statix signature generation based on SDF3 specification FlowSpec Stratego 2 Transformation (compilation) Stratego 2 Testing SPT Not all expectations have been ported over yet Editor services Syntax-based styling Inline error/warning/note messages Reference resolution Hover tooltips Code completion Syntactic Semantic (i.e., based on static semantics) Outline Platforms Command-line Eclipse Concurrency/parallelism is mostly ignored. Therefore, things may run concurrently that are not suppose to which cause data races and crashes. Several editor services and other conveniences are still missing or work in progress. IntelliJ A very minimal IntelliJ plugin for your language is provided, currently only supporting syntax highlighting and inline parse errors. Gradle Maven REPL The following features are being prototyped/experimented with Spoofax 3: Multi-lingual semantic analysis with Statix (Aron Zwaan) Semantic code completion based on Statix specification (Daniel Pelsmaeker) The following features will most likely not be supported: Analysis with NaBL/TS","title":"Current Status"},{"location":"guide/ask_for_help/","text":"Ask for help \u00b6 Before asking for help: try to troubleshoot the problem yourself . Performing troubleshooting makes you more familiar with the logging features of Spoofax, which can help you to improve the question you ask. check if the question has already been asked and/or answered in the Spoofax 3 Ask a question discussion forum by searching for your question. If not answered, continue discussion on that question. check if there is a bug report related to your question in the Spoofax 3 issue tracker by searching. Also check closed issues, as they may provide an answer to your question. If there is an open issue related to your question, continue discussion on that issue. Ask for help by creating a new discussion in the Spoofax 3 Ask a question discussion forum . Comprehensive and detailed questions are more likely to be answered. Try to include the following information in your question if applicable: Spoofax version Operating system & version Errors and stack traces (if applicable) Build logs (if applicable) Thread dumps (if applicable) Heap dumps (if applicable) Java version (if not using an embedded JVM) Eclipse version (if not using the standard Eclipse installation) Gradle version (if applicable) Also, please consider helping other members by answering questions if you know the answer!","title":"Ask for help"},{"location":"guide/ask_for_help/#ask-for-help","text":"Before asking for help: try to troubleshoot the problem yourself . Performing troubleshooting makes you more familiar with the logging features of Spoofax, which can help you to improve the question you ask. check if the question has already been asked and/or answered in the Spoofax 3 Ask a question discussion forum by searching for your question. If not answered, continue discussion on that question. check if there is a bug report related to your question in the Spoofax 3 issue tracker by searching. Also check closed issues, as they may provide an answer to your question. If there is an open issue related to your question, continue discussion on that issue. Ask for help by creating a new discussion in the Spoofax 3 Ask a question discussion forum . Comprehensive and detailed questions are more likely to be answered. Try to include the following information in your question if applicable: Spoofax version Operating system & version Errors and stack traces (if applicable) Build logs (if applicable) Thread dumps (if applicable) Heap dumps (if applicable) Java version (if not using an embedded JVM) Eclipse version (if not using the standard Eclipse installation) Gradle version (if applicable) Also, please consider helping other members by answering questions if you know the answer!","title":"Ask for help"},{"location":"guide/report_a_bug/","text":"Report a bug \u00b6 Before reporting a bug: check if there is an existing bug report in the Spoofax 3 issue tracker by searching. Also check closed issues, as the bug may already be fixed or closed for other reasons. If there is already an open issue for the bug, continue discussion on that issue. Report the bug by creating a new issue in the Spoofax 3 issue tracker . Make sure to fill in the template, and to be comprehensive and detailed, to make it easier for us to resolve the bug report. The troubleshooting guide can help you with getting exception stack traces, logs, thread dumps, and heap dumps. Please include these with the bug report if you think they are applicable.","title":"Report a bug"},{"location":"guide/report_a_bug/#report-a-bug","text":"Before reporting a bug: check if there is an existing bug report in the Spoofax 3 issue tracker by searching. Also check closed issues, as the bug may already be fixed or closed for other reasons. If there is already an open issue for the bug, continue discussion on that issue. Report the bug by creating a new issue in the Spoofax 3 issue tracker . Make sure to fill in the template, and to be comprehensive and detailed, to make it easier for us to resolve the bug report. The troubleshooting guide can help you with getting exception stack traces, logs, thread dumps, and heap dumps. Please include these with the bug report if you think they are applicable.","title":"Report a bug"},{"location":"guide/development/debugging-in-intellij/","text":"How to debug Spoofax 3 development using IntelliJ \u00b6 To debug Spoofax 3 development using IntelliJ, create a new Run Configuration and pass the --debug-jvm option to the task. For example: :spoofax3.example.root:tiger.eclipse:runEclipse --debug-jvm Error: Unknown command-line option '--debug-jvm' You specified the delegate task name (e.g., runTigerEclipse ) instead of the full task name. Only the JavaExec type tasks support the --debug-jvm option, and the delegate tasks are of the wrong type. Specify the full task name instead. For example, if the delegate task runTigerEclipse is defined in the root build.gradle.kts like this: build.gradle.kts tasksWithIncludedBuild ( \"spoofax3.example.root\" ) { registerDelegateTask ( \"runTigerCli\" , it , \":tiger.cli:run\" ) registerDelegateTask ( \"runTigerEclipse\" , it , \":tiger.eclipse:runEclipse\" ) registerDelegateTask ( \"runTigerIntelliJ\" , it , \":tiger.intellij:runIde\" ) } Then the full task name is the name of the included build combined with the name of the task, starting and separated with colons ( : ). Thus it would be: :spoofax3.example.root:tiger.eclipse:runEclipse Then start the configuration in Run mode, and wait until the following shows up in the console: Listening for transport dt_socket at address: 5005 [Attach debugger] Then click the Attach debugger text in the console to start and attach an external debugger and start debugging. Debugging tests \u00b6 If you are debugging tests, make sure that the test results are cleaned before by running cleanTest , otherwise Gradle may skip the test task. For example, run the following Gradle tasks as part of the Run configuration: :spoofax3.lwb.root:spoofax.dynamicloading:cleanTest :spoofax3.lwb.root:spoofax.dynamicloading:test What about JAVA_TOOL_OPTIONS ? \u00b6 Instead of specifying --debug-jvm , you can add the following configuration variable to your Run configuration: JAVA_TOOL_OPTIONS=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 This will work regardless of which task is specified, including specifying a delegate (non JavaExec ) task. However, note an important downside with this approach: this enables debugging for any Gradle task that executes Java in an isolated way, including any (Java/Kotlin) compilation tasks that run in a separate process. You might see the following error: Picked up JAVA_TOOL_OPTIONS: -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 ERROR: transport error 202: bind failed: Address already in use FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197) ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [debugInit.c:750] To avoid this error, make sure to first build normally such that these tasks are no longer executed, then run your debugging configuration.","title":"Debugging in IntelliJ"},{"location":"guide/development/debugging-in-intellij/#how-to-debug-spoofax-3-development-using-intellij","text":"To debug Spoofax 3 development using IntelliJ, create a new Run Configuration and pass the --debug-jvm option to the task. For example: :spoofax3.example.root:tiger.eclipse:runEclipse --debug-jvm Error: Unknown command-line option '--debug-jvm' You specified the delegate task name (e.g., runTigerEclipse ) instead of the full task name. Only the JavaExec type tasks support the --debug-jvm option, and the delegate tasks are of the wrong type. Specify the full task name instead. For example, if the delegate task runTigerEclipse is defined in the root build.gradle.kts like this: build.gradle.kts tasksWithIncludedBuild ( \"spoofax3.example.root\" ) { registerDelegateTask ( \"runTigerCli\" , it , \":tiger.cli:run\" ) registerDelegateTask ( \"runTigerEclipse\" , it , \":tiger.eclipse:runEclipse\" ) registerDelegateTask ( \"runTigerIntelliJ\" , it , \":tiger.intellij:runIde\" ) } Then the full task name is the name of the included build combined with the name of the task, starting and separated with colons ( : ). Thus it would be: :spoofax3.example.root:tiger.eclipse:runEclipse Then start the configuration in Run mode, and wait until the following shows up in the console: Listening for transport dt_socket at address: 5005 [Attach debugger] Then click the Attach debugger text in the console to start and attach an external debugger and start debugging.","title":"How to debug Spoofax 3 development using IntelliJ"},{"location":"guide/development/debugging-in-intellij/#debugging-tests","text":"If you are debugging tests, make sure that the test results are cleaned before by running cleanTest , otherwise Gradle may skip the test task. For example, run the following Gradle tasks as part of the Run configuration: :spoofax3.lwb.root:spoofax.dynamicloading:cleanTest :spoofax3.lwb.root:spoofax.dynamicloading:test","title":"Debugging tests"},{"location":"guide/development/debugging-in-intellij/#what-about-java_tool_options","text":"Instead of specifying --debug-jvm , you can add the following configuration variable to your Run configuration: JAVA_TOOL_OPTIONS=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 This will work regardless of which task is specified, including specifying a delegate (non JavaExec ) task. However, note an important downside with this approach: this enables debugging for any Gradle task that executes Java in an isolated way, including any (Java/Kotlin) compilation tasks that run in a separate process. You might see the following error: Picked up JAVA_TOOL_OPTIONS: -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 ERROR: transport error 202: bind failed: Address already in use FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197) ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [debugInit.c:750] To avoid this error, make sure to first build normally such that these tasks are no longer executed, then run your debugging configuration.","title":"What about JAVA_TOOL_OPTIONS?"},{"location":"guide/development/troubleshooting/","text":"Troubleshooting \u00b6 This guide explains how to troubleshoot issues when developing Spoofax itself. In general, ensure you're calling ./repo and ./gradlew on Linux and MacOS (or repo.bat and gradlew.bat on Windows) instead of your local Gradle installation. The local one may be too old or too new. Known Problems \u00b6 The following are known problems that can occur, and their solutions or workarounds. Many errors about unresolved classes in IntelliJ \u00b6 If after importing there are many errors in files about classes not existing, re-import all the projects by pressing the Reload All Gradle Projects button in the Gradle tool window. Cannot debug in IntelliJ \u00b6 See how to debug in IntelliJ for more information and tips. Profiling in IntelliJ \u00b6 Profiling in IntelliJ can be done similarly to debugging. For example, to profile with YourKit, add the following environment variable to your run configuration: JAVA_TOOL_OPTIONS=-agentpath:/Applications/YourKit-Java-Profiler-2020.9.app/Contents/Resources/bin/mac/libyjpagent.dylib=listen=all,sampling,onexit=snapshot If you are using a different profiler, the agentpath needs to point to the corresponding agent of your profiler, and the settings after the agent will need to be tailored towards your profiler. In the example above, the YourKit profiler will attach to the program, enable CPU sampling, and create a snapshot when the program ends. The snapshot can then be opened and inspected in YourKit. Similar to debugging, this enables profiling for any Gradle task that executes Java in an isolated way, and tests must be cleaned before profiling to force tests to be executed. Spoofax 2 language fails to build with \"Previous build failed and no change in the build input has been observed\" \u00b6 If building a Spoofax 2 language fails due to some ephemeral issue, or if building is cancelled (because you cancelled the Gradle build), the following exception may be thrown during the build: org.metaborg.core.MetaborgException: Previous build failed and no change in the build input has been observed, not rebuilding. Fix the problem, or clean and rebuild the project to force a rebuild This is an artefact of the Pluto build system refusing to rebuild if it failed but no changes to the input were detected. To force Pluto to rebuild, delete the target/pluto directory of the language. Task 'buildAll' not found in root project 'devenv' \u00b6 You have 'configure on demand' enabled, such as org.gradle.configureondemand=true in your ~/.gradle/gradle.properties file. Disable this. Expiring Daemon because JVM heap space is exhausted \u00b6 The memory limits in gradle.properties may be too low, and may need to be increased. Running the build without --parallel may decrease memory pressure, as less tasks are executed concurrently. Or, there is a memory leak in the build: please make a heap dump and send this to the developers so it can be addressed. Could not create service of type FileAccessTimeJournal using GradleUserHomeScopeServices.createFileAccessTimeJournal() \u00b6 The permissions in your ~/.gradle/ directory are too restrictive. For example, if you're using WSL, ensure the directory is not a symlink to the Windows' .gradle/ directory. Error resolving plugin: Plugin request for plugin already on the classpath must not include a version \u00b6 Error resolving plugin [id: 'org.metaborg.gradle.config.devenv', version: '?'] Plugin request for plugin already on the classpath must not include a version You are not running with the recommended version of Gradle. Unknown command-line option '--args' \u00b6 Command-line arguments such as --args are not supported for tasks in the root project, such as the runSdf3Cli task. Instead, go to the relevant included build and call the task directly. cd spoofax.pie/example ./gradlew :sdf3.cli:run --args=\"-V\" The working directory is the directory with the gradle.build.kts file of the CLI project. This cannot be changed. For example, spoofax.pie/example/sdf3/sdf3.cli/ for the :sdf3.cli project. Failed to create Jar file \u00b6 Due to a bug in the Java 8 and 9 compiler, it sometimes generates the wrong byte code for annotations . This, in turn, causes Gradle to fail reading the JAR file with the following error: A problem occurred configuring project ':spoofax3.example.root'. > Failed to create Jar file /Users/username/.gradle/caches/jars-8/defabc/jsglr.common-develop-SNAPSHOT.jar. The stack trace will look something like this: org.gradle.api.ProjectConfigurationException: A problem occurred configuring project ':spoofax3.example.root'. at org.gradle.configuration.project.LifecycleProjectEvaluator.wrapException(LifecycleProjectEvaluator.java:75) at org.gradle.configuration.project.LifecycleProjectEvaluator.addConfigurationFailure(LifecycleProjectEvaluator.java:68) at org.gradle.configuration.project.LifecycleProjectEvaluator.access$400(LifecycleProjectEvaluator.java:51) ... 142 more Caused by: org.gradle.api.GradleException: Failed to create Jar file /Users/username/.gradle/caches/jars-8/defabc/jsglr.common-develop-SNAPSHOT.jar. at org.gradle.internal.classpath.ClasspathBuilder.jar(ClasspathBuilder.java:47) at org.gradle.internal.classpath.InstrumentingClasspathFileTransformer.instrument(InstrumentingClasspathFileTransformer.java:83) ... 6 more Caused by: java.lang.ArrayIndexOutOfBoundsException: 195 at org.objectweb.asm.ClassReader.readLabel(ClassReader.java:2654) at org.objectweb.asm.ClassReader.createLabel(ClassReader.java:2670) at org.objectweb.asm.ClassReader.readTypeAnnotations(ClassReader.java:2736) at org.objectweb.asm.ClassReader.readCode(ClassReader.java:1912) at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1492) at org.objectweb.asm.ClassReader.accept(ClassReader.java:717) ... 15 more Determine the version of Java using java -version . Java 8 and 9 can exhibit this problem. The solution is to update your Java to version 11 or later. You can use a tool such as SDKMAN! to easily manage the (default) versions of Java on your system.","title":"Troubleshooting"},{"location":"guide/development/troubleshooting/#troubleshooting","text":"This guide explains how to troubleshoot issues when developing Spoofax itself. In general, ensure you're calling ./repo and ./gradlew on Linux and MacOS (or repo.bat and gradlew.bat on Windows) instead of your local Gradle installation. The local one may be too old or too new.","title":"Troubleshooting"},{"location":"guide/development/troubleshooting/#known-problems","text":"The following are known problems that can occur, and their solutions or workarounds.","title":"Known Problems"},{"location":"guide/development/troubleshooting/#many-errors-about-unresolved-classes-in-intellij","text":"If after importing there are many errors in files about classes not existing, re-import all the projects by pressing the Reload All Gradle Projects button in the Gradle tool window.","title":"Many errors about unresolved classes in IntelliJ"},{"location":"guide/development/troubleshooting/#cannot-debug-in-intellij","text":"See how to debug in IntelliJ for more information and tips.","title":"Cannot debug in IntelliJ"},{"location":"guide/development/troubleshooting/#profiling-in-intellij","text":"Profiling in IntelliJ can be done similarly to debugging. For example, to profile with YourKit, add the following environment variable to your run configuration: JAVA_TOOL_OPTIONS=-agentpath:/Applications/YourKit-Java-Profiler-2020.9.app/Contents/Resources/bin/mac/libyjpagent.dylib=listen=all,sampling,onexit=snapshot If you are using a different profiler, the agentpath needs to point to the corresponding agent of your profiler, and the settings after the agent will need to be tailored towards your profiler. In the example above, the YourKit profiler will attach to the program, enable CPU sampling, and create a snapshot when the program ends. The snapshot can then be opened and inspected in YourKit. Similar to debugging, this enables profiling for any Gradle task that executes Java in an isolated way, and tests must be cleaned before profiling to force tests to be executed.","title":"Profiling in IntelliJ"},{"location":"guide/development/troubleshooting/#spoofax-2-language-fails-to-build-with-previous-build-failed-and-no-change-in-the-build-input-has-been-observed","text":"If building a Spoofax 2 language fails due to some ephemeral issue, or if building is cancelled (because you cancelled the Gradle build), the following exception may be thrown during the build: org.metaborg.core.MetaborgException: Previous build failed and no change in the build input has been observed, not rebuilding. Fix the problem, or clean and rebuild the project to force a rebuild This is an artefact of the Pluto build system refusing to rebuild if it failed but no changes to the input were detected. To force Pluto to rebuild, delete the target/pluto directory of the language.","title":"Spoofax 2 language fails to build with \"Previous build failed and no change in the build input has been observed\""},{"location":"guide/development/troubleshooting/#task-buildall-not-found-in-root-project-devenv","text":"You have 'configure on demand' enabled, such as org.gradle.configureondemand=true in your ~/.gradle/gradle.properties file. Disable this.","title":"Task 'buildAll' not found in root project 'devenv'"},{"location":"guide/development/troubleshooting/#expiring-daemon-because-jvm-heap-space-is-exhausted","text":"The memory limits in gradle.properties may be too low, and may need to be increased. Running the build without --parallel may decrease memory pressure, as less tasks are executed concurrently. Or, there is a memory leak in the build: please make a heap dump and send this to the developers so it can be addressed.","title":"Expiring Daemon because JVM heap space is exhausted"},{"location":"guide/development/troubleshooting/#could-not-create-service-of-type-fileaccesstimejournal-using-gradleuserhomescopeservicescreatefileaccesstimejournal","text":"The permissions in your ~/.gradle/ directory are too restrictive. For example, if you're using WSL, ensure the directory is not a symlink to the Windows' .gradle/ directory.","title":"Could not create service of type FileAccessTimeJournal using GradleUserHomeScopeServices.createFileAccessTimeJournal()"},{"location":"guide/development/troubleshooting/#error-resolving-plugin-plugin-request-for-plugin-already-on-the-classpath-must-not-include-a-version","text":"Error resolving plugin [id: 'org.metaborg.gradle.config.devenv', version: '?'] Plugin request for plugin already on the classpath must not include a version You are not running with the recommended version of Gradle.","title":"Error resolving plugin: Plugin request for plugin already on the classpath must not include a version"},{"location":"guide/development/troubleshooting/#unknown-command-line-option-args","text":"Command-line arguments such as --args are not supported for tasks in the root project, such as the runSdf3Cli task. Instead, go to the relevant included build and call the task directly. cd spoofax.pie/example ./gradlew :sdf3.cli:run --args=\"-V\" The working directory is the directory with the gradle.build.kts file of the CLI project. This cannot be changed. For example, spoofax.pie/example/sdf3/sdf3.cli/ for the :sdf3.cli project.","title":"Unknown command-line option '--args'"},{"location":"guide/development/troubleshooting/#failed-to-create-jar-file","text":"Due to a bug in the Java 8 and 9 compiler, it sometimes generates the wrong byte code for annotations . This, in turn, causes Gradle to fail reading the JAR file with the following error: A problem occurred configuring project ':spoofax3.example.root'. > Failed to create Jar file /Users/username/.gradle/caches/jars-8/defabc/jsglr.common-develop-SNAPSHOT.jar. The stack trace will look something like this: org.gradle.api.ProjectConfigurationException: A problem occurred configuring project ':spoofax3.example.root'. at org.gradle.configuration.project.LifecycleProjectEvaluator.wrapException(LifecycleProjectEvaluator.java:75) at org.gradle.configuration.project.LifecycleProjectEvaluator.addConfigurationFailure(LifecycleProjectEvaluator.java:68) at org.gradle.configuration.project.LifecycleProjectEvaluator.access$400(LifecycleProjectEvaluator.java:51) ... 142 more Caused by: org.gradle.api.GradleException: Failed to create Jar file /Users/username/.gradle/caches/jars-8/defabc/jsglr.common-develop-SNAPSHOT.jar. at org.gradle.internal.classpath.ClasspathBuilder.jar(ClasspathBuilder.java:47) at org.gradle.internal.classpath.InstrumentingClasspathFileTransformer.instrument(InstrumentingClasspathFileTransformer.java:83) ... 6 more Caused by: java.lang.ArrayIndexOutOfBoundsException: 195 at org.objectweb.asm.ClassReader.readLabel(ClassReader.java:2654) at org.objectweb.asm.ClassReader.createLabel(ClassReader.java:2670) at org.objectweb.asm.ClassReader.readTypeAnnotations(ClassReader.java:2736) at org.objectweb.asm.ClassReader.readCode(ClassReader.java:1912) at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1492) at org.objectweb.asm.ClassReader.accept(ClassReader.java:717) ... 15 more Determine the version of Java using java -version . Java 8 and 9 can exhibit this problem. The solution is to update your Java to version 11 or later. You can use a tool such as SDKMAN! to easily manage the (default) versions of Java on your system.","title":"Failed to create Jar file"},{"location":"guide/eclipse_lwb/import/","text":"Importing a Project \u00b6 An existing Spoofax 3 language project can be imported into Eclipse as follows. In the main menu of Eclipse, select File \u2023 Import... to open the import dialog. In the import dialog, select General \u2023 Existing Projects into Workspace and press Next > to open the import projects dialog. In the import projects dialog, ensure Select root directory: is ticked and press Browse... on the right of that. Select the root directory of the language project that you want to import (the directory that contains spoofaxc.cfg ) and press Open . Then, press Finish to import the project. The project should now be imported into your workspace. Finally, build the language project by selecting the project in the Package Explorer and choosing Project \u2023 Build Project . Building the project at least once after importing is required to update the project. Troubleshooting \u00b6 The project does not appear in the Import window \u00b6 Ensure the project has associated .project and .classpath files, and that they are not ignored for version control. Minimum file content .","title":"Importing a Project"},{"location":"guide/eclipse_lwb/import/#importing-a-project","text":"An existing Spoofax 3 language project can be imported into Eclipse as follows. In the main menu of Eclipse, select File \u2023 Import... to open the import dialog. In the import dialog, select General \u2023 Existing Projects into Workspace and press Next > to open the import projects dialog. In the import projects dialog, ensure Select root directory: is ticked and press Browse... on the right of that. Select the root directory of the language project that you want to import (the directory that contains spoofaxc.cfg ) and press Open . Then, press Finish to import the project. The project should now be imported into your workspace. Finally, build the language project by selecting the project in the Package Explorer and choosing Project \u2023 Build Project . Building the project at least once after importing is required to update the project.","title":"Importing a Project"},{"location":"guide/eclipse_lwb/import/#troubleshooting","text":"","title":"Troubleshooting"},{"location":"guide/eclipse_lwb/import/#the-project-does-not-appear-in-the-import-window","text":"Ensure the project has associated .project and .classpath files, and that they are not ignored for version control. Minimum file content .","title":"The project does not appear in the Import window"},{"location":"guide/eclipse_lwb/troubleshooting/","text":"Troubleshooting \u00b6 This how-to guide explains how to troubleshoot issues when building and testing languages in the Spoofax Eclipse LWB plugin. Diagnosing problems \u00b6 Looking for errors \u00b6 If something is not working as expected, the first thing to check is whether there are errors in the project by looking in the Project Explorer or Package Explorer view, which is open on the left-hand side of the IDE by default. If there are any red markers on the project, or any directory or file in the project, that indicates an error. When there are errors on files that define your language (e.g., .cfg/.sdf3/.stx/.str2 files), the language is not compiled and reloaded, so building the language has no effect, and your change will not be taken into account. Errors on example and test (e.g., .spt) files are ok, and do not prevent the language from being compiled and reloaded. It is also possible to get a list of all the errors in the project by opening the Problems view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Problems from the main menu. Consulting the logs \u00b6 Spoofax logs a lot of information to make troubleshooting easier. First consult the Error Log view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Error Log from the main menu. The error log contains warning and error events from all plugins running in Eclipse, including several Spoofax plugins. Most errors include stack traces, which help the Spoofax developers immensely in bug reports or when asking for help. Browse through the errors from Spoofax plugins to see if it can help you troubleshoot the problem. If nothing relevant is in the error log, try to consult the Console view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Other... \u2023 search for Console \u2023 choose Console and open it from the main menu. If the console view does not say Spoofax in the top-left corner, ensure the Spoofax console is open by pressing the small downward error on the blue monitor icon and selecting Spoofax console (see screenshot). Browse through the log from Spoofax plugins to see if it can help you troubleshoot the problem. When building, Spoofax logs all tasks and files that are checked, and all tasks that are built. At the end of the build, Spoofax logs whether it completed or if something went wrong. Rebuild your language and check the build log to see if it can help you troubleshoot the problem. If you ask for help or report a bug, consider storing the log in a text file, so you can include it in your help request or bug report. Problems and solutions \u00b6 Errors in language definition files \u00b6 Error markers in language definition files (e.g., .cfg/.sdf3/.stx/.str2 files) are expected. If there are errors on language definition files, solve them build the language. Errors in generated files \u00b6 If there are errors in generated files (i.e., files in the build/generated directory), first try to build the language, as building the language may re-generate these files. If the error persists, try deleting the generated file and then building the language. Also consider reporting this bug , as we consider incrementality issues like these bugs. If after deleting the generated file and rebuilding the language, the generated file comes back with errors, definitely report this as a bug . Try to work around the problem first by cleaning the language project or by deleting on-disk cache and restarting Eclipse Errors occurred during the build \u00b6 If after building a language, an error popup appears, something unexpected went wrong. Please report this as a bug . Try to work around the problem first by cleaning the language project or by deleting on-disk cache and restarting Eclipse Many duplicate definition errors/other weird errors \u00b6 If there are many errors about duplicate definitions, things already being defined, or other weird errors, see if there is a bin directory in your language project. If so, delete the bin directory and rebuild the project. Workarounds \u00b6 Cleaning the language project \u00b6 Try cleaning the language project by first selecting (clicking) the language project in the Project Explorer or Package Explorer view, and then choosing Project \u2023 Clean... from the main menu. In the clean window, deselect Clean all projects , select your project, and press Clean . Then, rebuild your language. Optionally, check the error log and console again. If this does not help, you can try to first delete the build directory, then clean the language project, and then build the project. If this solves your problem, consider reporting this bug , as we consider incrementality issues like these bugs. If not, try the next workaround. Deleting on-disk cache and restarting Eclipse \u00b6 Finally, there may be a problem related to the on-disk cache. First, close Eclipse. Then, navigate to your workspace directory and delete the .metadata/.plugins/spoofax.lwb.eclipse/pieStore file. These directories are hidden, so you may need to enable showing hidden files, or delete the file using your terminal. Then, start Eclipse again and build your language. Optionally, check the error log and console again. If this solves your problem, consider reporting this bug , as we consider incrementality issues like these bugs. If not, ask for help . Report a bug or ask for help \u00b6 If after troubleshooting the issue is not resolved, report this bug if you think this is a bug, or ask for help . Advanced troubleshooting \u00b6 Checking for deadlocks (and making a thread dump) \u00b6 If Eclipse seems to be stuck, hanging, or not making any progress, check for deadlocks by making a thread dump. First, we need to figure out the process ID of the JVM that is running eclipse. Run the jps command in a terminal. It will print something like: 80035 Jps 94805 74294 org.eclipse.equinox.launcher_1.6.100.v20201223-0822.jar 51515 Eclipse 74236 GradleDaemon In this case, 51515 is the process ID of Eclipse. Then run jstack , so jstack 51515 in this case. jstack prints a thread dump with a stack trace for each thread, alongside any detected deadlocks. While it may be hard to use this information to troubleshoot yourself, this can be useful information when asking for help or when reporting a bug. In case of deadlocks, please report this bug . Checking memory (and making a heap dump) \u00b6 If Eclipse seems to be using excessive amounts of memory or processor time, check how much heap space Eclipse is using. To show the heap space Eclipse is using, go to the Eclipse preferences, and in the General tab, enable Show heap status . The heap status shows up in the bottom right corner. Press the trash can icon to run garbage collection, which will free up any available memory. If after garbage collection, the memory is still near its maximum, Eclipse has run out of memory and will become very slow or unresponsive. To diagnose the problem, first make a thread dump as was described in the previous section. A thread dump may give some clues as to what is generating or leaking heap memory. Then, make a heap dump by running jmap -dump:live,format=b,file=heap.bin using the process ID of Eclipse, as was described in the previous section. This creates a heap dump file called heap.bin in the working directory. This heap dump can then be loaded into a profiler such as VisualVM for inspection. In case of excessive memory problems, please report this bug and share the heap dump. To share heap dumps, upload them to a cloud service such as Mega and share the link. Warning Heap dumps contain all the JVM memory in Eclipse, which can include any information that you have entered into Eclipse. Only share a heap dump if you are sure you have not entered any sensitive information into Eclipse.","title":"Troubleshooting"},{"location":"guide/eclipse_lwb/troubleshooting/#troubleshooting","text":"This how-to guide explains how to troubleshoot issues when building and testing languages in the Spoofax Eclipse LWB plugin.","title":"Troubleshooting"},{"location":"guide/eclipse_lwb/troubleshooting/#diagnosing-problems","text":"","title":"Diagnosing problems"},{"location":"guide/eclipse_lwb/troubleshooting/#looking-for-errors","text":"If something is not working as expected, the first thing to check is whether there are errors in the project by looking in the Project Explorer or Package Explorer view, which is open on the left-hand side of the IDE by default. If there are any red markers on the project, or any directory or file in the project, that indicates an error. When there are errors on files that define your language (e.g., .cfg/.sdf3/.stx/.str2 files), the language is not compiled and reloaded, so building the language has no effect, and your change will not be taken into account. Errors on example and test (e.g., .spt) files are ok, and do not prevent the language from being compiled and reloaded. It is also possible to get a list of all the errors in the project by opening the Problems view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Problems from the main menu.","title":"Looking for errors"},{"location":"guide/eclipse_lwb/troubleshooting/#consulting-the-logs","text":"Spoofax logs a lot of information to make troubleshooting easier. First consult the Error Log view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Error Log from the main menu. The error log contains warning and error events from all plugins running in Eclipse, including several Spoofax plugins. Most errors include stack traces, which help the Spoofax developers immensely in bug reports or when asking for help. Browse through the errors from Spoofax plugins to see if it can help you troubleshoot the problem. If nothing relevant is in the error log, try to consult the Console view. If this view is not open, open it by choosing Window \u2023 Show view \u2023 Other... \u2023 search for Console \u2023 choose Console and open it from the main menu. If the console view does not say Spoofax in the top-left corner, ensure the Spoofax console is open by pressing the small downward error on the blue monitor icon and selecting Spoofax console (see screenshot). Browse through the log from Spoofax plugins to see if it can help you troubleshoot the problem. When building, Spoofax logs all tasks and files that are checked, and all tasks that are built. At the end of the build, Spoofax logs whether it completed or if something went wrong. Rebuild your language and check the build log to see if it can help you troubleshoot the problem. If you ask for help or report a bug, consider storing the log in a text file, so you can include it in your help request or bug report.","title":"Consulting the logs"},{"location":"guide/eclipse_lwb/troubleshooting/#problems-and-solutions","text":"","title":"Problems and solutions"},{"location":"guide/eclipse_lwb/troubleshooting/#errors-in-language-definition-files","text":"Error markers in language definition files (e.g., .cfg/.sdf3/.stx/.str2 files) are expected. If there are errors on language definition files, solve them build the language.","title":"Errors in language definition files"},{"location":"guide/eclipse_lwb/troubleshooting/#errors-in-generated-files","text":"If there are errors in generated files (i.e., files in the build/generated directory), first try to build the language, as building the language may re-generate these files. If the error persists, try deleting the generated file and then building the language. Also consider reporting this bug , as we consider incrementality issues like these bugs. If after deleting the generated file and rebuilding the language, the generated file comes back with errors, definitely report this as a bug . Try to work around the problem first by cleaning the language project or by deleting on-disk cache and restarting Eclipse","title":"Errors in generated files"},{"location":"guide/eclipse_lwb/troubleshooting/#errors-occurred-during-the-build","text":"If after building a language, an error popup appears, something unexpected went wrong. Please report this as a bug . Try to work around the problem first by cleaning the language project or by deleting on-disk cache and restarting Eclipse","title":"Errors occurred during the build"},{"location":"guide/eclipse_lwb/troubleshooting/#many-duplicate-definition-errorsother-weird-errors","text":"If there are many errors about duplicate definitions, things already being defined, or other weird errors, see if there is a bin directory in your language project. If so, delete the bin directory and rebuild the project.","title":"Many duplicate definition errors/other weird errors"},{"location":"guide/eclipse_lwb/troubleshooting/#workarounds","text":"","title":"Workarounds"},{"location":"guide/eclipse_lwb/troubleshooting/#clean_project","text":"Try cleaning the language project by first selecting (clicking) the language project in the Project Explorer or Package Explorer view, and then choosing Project \u2023 Clean... from the main menu. In the clean window, deselect Clean all projects , select your project, and press Clean . Then, rebuild your language. Optionally, check the error log and console again. If this does not help, you can try to first delete the build directory, then clean the language project, and then build the project. If this solves your problem, consider reporting this bug , as we consider incrementality issues like these bugs. If not, try the next workaround.","title":"Cleaning the language project"},{"location":"guide/eclipse_lwb/troubleshooting/#clean_cache","text":"Finally, there may be a problem related to the on-disk cache. First, close Eclipse. Then, navigate to your workspace directory and delete the .metadata/.plugins/spoofax.lwb.eclipse/pieStore file. These directories are hidden, so you may need to enable showing hidden files, or delete the file using your terminal. Then, start Eclipse again and build your language. Optionally, check the error log and console again. If this solves your problem, consider reporting this bug , as we consider incrementality issues like these bugs. If not, ask for help .","title":"Deleting on-disk cache and restarting Eclipse"},{"location":"guide/eclipse_lwb/troubleshooting/#report-a-bug-or-ask-for-help","text":"If after troubleshooting the issue is not resolved, report this bug if you think this is a bug, or ask for help .","title":"Report a bug or ask for help"},{"location":"guide/eclipse_lwb/troubleshooting/#advanced-troubleshooting","text":"","title":"Advanced troubleshooting"},{"location":"guide/eclipse_lwb/troubleshooting/#checking-for-deadlocks-and-making-a-thread-dump","text":"If Eclipse seems to be stuck, hanging, or not making any progress, check for deadlocks by making a thread dump. First, we need to figure out the process ID of the JVM that is running eclipse. Run the jps command in a terminal. It will print something like: 80035 Jps 94805 74294 org.eclipse.equinox.launcher_1.6.100.v20201223-0822.jar 51515 Eclipse 74236 GradleDaemon In this case, 51515 is the process ID of Eclipse. Then run jstack , so jstack 51515 in this case. jstack prints a thread dump with a stack trace for each thread, alongside any detected deadlocks. While it may be hard to use this information to troubleshoot yourself, this can be useful information when asking for help or when reporting a bug. In case of deadlocks, please report this bug .","title":"Checking for deadlocks (and making a thread dump)"},{"location":"guide/eclipse_lwb/troubleshooting/#checking-memory-and-making-a-heap-dump","text":"If Eclipse seems to be using excessive amounts of memory or processor time, check how much heap space Eclipse is using. To show the heap space Eclipse is using, go to the Eclipse preferences, and in the General tab, enable Show heap status . The heap status shows up in the bottom right corner. Press the trash can icon to run garbage collection, which will free up any available memory. If after garbage collection, the memory is still near its maximum, Eclipse has run out of memory and will become very slow or unresponsive. To diagnose the problem, first make a thread dump as was described in the previous section. A thread dump may give some clues as to what is generating or leaking heap memory. Then, make a heap dump by running jmap -dump:live,format=b,file=heap.bin using the process ID of Eclipse, as was described in the previous section. This creates a heap dump file called heap.bin in the working directory. This heap dump can then be loaded into a profiler such as VisualVM for inspection. In case of excessive memory problems, please report this bug and share the heap dump. To share heap dumps, upload them to a cloud service such as Mega and share the link. Warning Heap dumps contain all the JVM memory in Eclipse, which can include any information that you have entered into Eclipse. Only share a heap dump if you are sure you have not entered any sensitive information into Eclipse.","title":"Checking memory (and making a heap dump)"},{"location":"guide/eclipse_lwb/update/","text":"Updating & Downgrading \u00b6 Updating \u00b6 The Spoofax 3 language workbench Eclipse plugin can be updated as follows. In the main menu of Eclipse, select Help \u2023 Install New Software... to open the install dialog. In this dialog, copy the update site of the version you want to install into the Work with: field and press Enter . If you want to install the latest released version, use this update site: https://artifacts.metaborg.org/content/unzip/releases-unzipped/org/metaborg/spoofax.lwb.eclipse.repository/0.22.0/spoofax.lwb.eclipse.repository-0.22.0.zip-unzip/ In the table below, check the checkbox next to Spoofax LWB and press Next > . A dialog will pop up with install details. Press Finish and wait for Eclipse to install the plugin. If a Security Warning pops up, press Install anyway , as Spoofax 3 is currently not signed. Finally, a dialog will pop up asking you to restart Eclipse. Press Restart Now to restart Eclipse. After restarting Eclipse, all projects must be cleaned in order to prevent compatibility issues. In the main menu of Eclipse, select Project \u2023 Clean... , check the Clean all project checkbox, and press Clean . Then, rebuild all projects with Project \u2023 Build All from the main menu, or by pressing Cmd + B on macOS or Ctrl + B on others. Downgrading \u00b6 Eclipse does not support directly downgrading to a previous version. Therefore, we must first uninstall the plugin. In the main menu, select Help \u2023 Install New Software... to open a new dialog. In this dialog, click the already installed link to open a list of all installed features. In that list, select Spoofax LWB and press Uninstall... . A dialog will pop up detailing the uninstall details. Press Finish and wait for Eclipse to uninstall the plugin. Finally, a dialog will pop up asking you to restart Eclipse. Press Restart Now to restart Eclipse. After Eclipse has restarted, follow the updating instructions above, but instead of using the latest update site, use the update site of an older version instead.","title":"Updating & Downgrading"},{"location":"guide/eclipse_lwb/update/#updating-downgrading","text":"","title":"Updating & Downgrading"},{"location":"guide/eclipse_lwb/update/#updating","text":"The Spoofax 3 language workbench Eclipse plugin can be updated as follows. In the main menu of Eclipse, select Help \u2023 Install New Software... to open the install dialog. In this dialog, copy the update site of the version you want to install into the Work with: field and press Enter . If you want to install the latest released version, use this update site: https://artifacts.metaborg.org/content/unzip/releases-unzipped/org/metaborg/spoofax.lwb.eclipse.repository/0.22.0/spoofax.lwb.eclipse.repository-0.22.0.zip-unzip/ In the table below, check the checkbox next to Spoofax LWB and press Next > . A dialog will pop up with install details. Press Finish and wait for Eclipse to install the plugin. If a Security Warning pops up, press Install anyway , as Spoofax 3 is currently not signed. Finally, a dialog will pop up asking you to restart Eclipse. Press Restart Now to restart Eclipse. After restarting Eclipse, all projects must be cleaned in order to prevent compatibility issues. In the main menu of Eclipse, select Project \u2023 Clean... , check the Clean all project checkbox, and press Clean . Then, rebuild all projects with Project \u2023 Build All from the main menu, or by pressing Cmd + B on macOS or Ctrl + B on others.","title":"Updating"},{"location":"guide/eclipse_lwb/update/#downgrading","text":"Eclipse does not support directly downgrading to a previous version. Therefore, we must first uninstall the plugin. In the main menu, select Help \u2023 Install New Software... to open a new dialog. In this dialog, click the already installed link to open a list of all installed features. In that list, select Spoofax LWB and press Uninstall... . A dialog will pop up detailing the uninstall details. Press Finish and wait for Eclipse to uninstall the plugin. Finally, a dialog will pop up asking you to restart Eclipse. Press Restart Now to restart Eclipse. After Eclipse has restarted, follow the updating instructions above, but instead of using the latest update site, use the update site of an older version instead.","title":"Downgrading"},{"location":"guide/static-semantics/code-completion/","text":"How to Enable Semantic Code Completion \u00b6 Semantic code completion is now part of Spoofax 3. Enabling Semantic Code Completion \u00b6 To enable support for semantic code completion in your language: Add the following to your language's spoofaxc.cfg file: tego-runtime {} code-completion {} Declare the following strategies in your language's main.str2 file, where MyLang is the name of your languages (as defined in spoofaxc.cfg : rules // Analysis downgrade-placeholders = downgrade-placeholders-MyLang upgrade-placeholders = upgrade-placeholders-MyLang is-inj = is-MyLang-inj-cons pp-partial = pp-partial-MyLang-string pre-analyze = explicate-injections-MyLang post-analyze = implicate-injections-MyLang In your Statix files, for each rule define a predicate that accepts a placeholder where a syntactic sort is permitted. For example: rules // placeholders programOk ( Module-Plhdr ()). declOk ( _ , Decl-Plhdr ()). typeOfExp ( _ , Exp-Plhdr ()) = _ . Using Semantic Code Completion \u00b6 In Eclipse, in a file of your language, type the placeholder somewhere where it is permitted. Unless overridden, the placeholder is a sort name within double square brackets, such as [[Exp]] . Then put the caret on the placeholder and press Cmd + Space on macOS (or Ctrl + Space on Linux/Windows) to invoke code completion. In a future release the placeholder will not need to be input explicitly. Limitations \u00b6 For this first release of semantic code completion, there are some limitations: You have to type the placeholder explicitly to invoke code completion. If the file contains errors, code completion might fail to return results. If you have catch-all predicates, code completion will not work. For example: // This prevents code completion from finding completions: typeOfExp ( _ , _ ) = _ : - try { false } | warning $[ This expression is not yet implemented ] . These limitations will be lifted in subsequent releases.","title":"Semantic Code Completion"},{"location":"guide/static-semantics/code-completion/#how-to-enable-semantic-code-completion","text":"Semantic code completion is now part of Spoofax 3.","title":"How to Enable Semantic Code Completion"},{"location":"guide/static-semantics/code-completion/#enabling-semantic-code-completion","text":"To enable support for semantic code completion in your language: Add the following to your language's spoofaxc.cfg file: tego-runtime {} code-completion {} Declare the following strategies in your language's main.str2 file, where MyLang is the name of your languages (as defined in spoofaxc.cfg : rules // Analysis downgrade-placeholders = downgrade-placeholders-MyLang upgrade-placeholders = upgrade-placeholders-MyLang is-inj = is-MyLang-inj-cons pp-partial = pp-partial-MyLang-string pre-analyze = explicate-injections-MyLang post-analyze = implicate-injections-MyLang In your Statix files, for each rule define a predicate that accepts a placeholder where a syntactic sort is permitted. For example: rules // placeholders programOk ( Module-Plhdr ()). declOk ( _ , Decl-Plhdr ()). typeOfExp ( _ , Exp-Plhdr ()) = _ .","title":"Enabling Semantic Code Completion"},{"location":"guide/static-semantics/code-completion/#using-semantic-code-completion","text":"In Eclipse, in a file of your language, type the placeholder somewhere where it is permitted. Unless overridden, the placeholder is a sort name within double square brackets, such as [[Exp]] . Then put the caret on the placeholder and press Cmd + Space on macOS (or Ctrl + Space on Linux/Windows) to invoke code completion. In a future release the placeholder will not need to be input explicitly.","title":"Using Semantic Code Completion"},{"location":"guide/static-semantics/code-completion/#limitations","text":"For this first release of semantic code completion, there are some limitations: You have to type the placeholder explicitly to invoke code completion. If the file contains errors, code completion might fail to return results. If you have catch-all predicates, code completion will not work. For example: // This prevents code completion from finding completions: typeOfExp ( _ , _ ) = _ : - try { false } | warning $[ This expression is not yet implemented ] . These limitations will be lifted in subsequent releases.","title":"Limitations"},{"location":"include/download/","text":"","title":"Download"},{"location":"reference/anatomy_language_implementation/","text":"Anatomy of a language implementation \u00b6 In this section we give a high-level overview of what a Spoofax 3 language implementation is, dive into details, and explain how such a implementation can be manually written or completely generated from a high-level language specification Overview \u00b6 In essence, a language implementation in Spoofax 3 is nothing more than a standard Java library (e.g., a JAR file) with Java classes implementing or delegating to the various functionalities of the language such as parsing and transformations, as well as bundled resources such as a parse table which is loaded and interpreted at runtime. Therefore, Spoofax 3 language implementations are very easy to use in the Java ecosystem by just distributing the JAR file of the language, or by publishing/consuming it as a library with a build system such as Gradle. Furthermore, since no classloading or class generation is used, GraalVM native image can be used to ahead-of-time compile your language implementation into native code which does not require a JVM at all, and significantly reduces the startup time of your language. Diving deeper, a language implementation is actually split into three parts: a language project that contains the base functionality of the language, an adapter project that adapts the language project to the interface of Spoofax, and platform projects that plug the adapter project into various other platforms such as a command-line interface (CLI) and Eclipse plugin (TODO: more details on supported platforms in a separate section). We will first explain these projects and why this separation was chosen. TODO: diagram? Language Project \u00b6 A language project contains the base functionality of a language, such as a parser, syntax highlighter, analyzer, and compiler for the language. Such a project is unstructured : it does not have to adhere to any interface or data format. Therefore, it may use any tooling, libraries, and data structures to implement the base functionality. This facilitates integration of existing tools and minimal dependencies. A language project is just a Java library and can thus be used in a standalone fashion. However, there is no glue between base functionality, requiring manual implementation of a parse-analyze-compile pipeline for example. Furthermore, because the project is unstructured, we cannot provide any integration with other platforms such as a CLI and Eclipse plugin. Therefore, using a language project as a standalone library is a bit of a niche use case for when minimal dependencies or full control is absolutely necessary. Because it is such a niche use case, the default in Spoofax 3 is to merge it together with the adapter project. In essence, an adapter project adapts a language project to Spoofax 3. To understand why, we first explain the high-level architecture of Spoofax 3. Spoofax 3 architecture overview \u00b6 Spoofax 3 provides a general interface for language implementations: LanguageInstance , which is used by platforms to automatically plug languages into their platform. For example, LanguageInstance has functionality for syntax highlighting , which when given a resource of the language, returns a syntax highlighting for that resource. (TODO: more details on the functionality in LanguageInstance in a separate section) Furthermore, Spoofax 3 uses PIE ; a framework for building incremental pipelines, build systems, and compilers; to incrementalize the language implementation. Instead of directly computing the syntax highlighting for a resource, we create a task that returns the syntax highlighting when demanded, with PIE taking care of whether it should recompute the syntax highlighting because the resource (or the syntax highlighting implementation) changed, or if it can just be returned from a cache. (TODO: more details on PIE in a separate section) A platform such as Eclipse or IntelliJ can then take a LanguageInstance implementation, demand the syntax highlighting task, and show the result it in the editor for your language. Therefore, any language that implements LanguageInstance can get syntax highlighting in Eclipse, IntelliJ, and any other supported platforms for free, with PIE taking care of coarse-grained incrementalization. To receive the benefits of Spoofax 3, the adapter project must thus be implemented for your language. Adapter Project \u00b6 An adapter project implements Spoofax 3's LanguageInstance using the language project. This requires glue code between the unstructured language project and the structured LanguageInstance interface. For example, you would need to convert the data structure that the syntax highlighter of your language returns, to one that Spoofax 3 understands: the Styling class. Furthermore, because Spoofax 3 uses PIE, we also need to implement a PIE task definition that implements the (re)computing of syntax highlighting, as well as mark all dependencies that should cause the syntax highlighting to be recomputed. We also need to be able to instantiate your implementation of LanguageInstance . In case this is non-trivial, the recommended practice is to use dependency injection to achieve proper separation of concerns. A dependency injection framework such as Dagger is recommended (we use it extensively in Spoofax 3) because it catches dependency injection errors at compile-time, and does not require runtime class loading or generation. This may sound like you would need to write a lot of boilerplate. However, we provide a compiler that generates all this boilerplate for you. It is only necessary to write this boilerplate if you are integrating existing tooling. Even then, the compiler can generate some of the boilerplate for you. More details on the compiler can be found in the developing language implementations section. Platform Projects \u00b6 TODO: every language-platform combination is a separate project to support ahead-of-time compilation, static loading, and customization of the platform project. CLI: can be ahead-of-time compiled with GraalVM native image to create a native Windows/macOS/Linux CLI for your language Eclipse/IntelliJ: statically loaded plugin that can be deployed with Eclipse/IntelliJ, and can be fully customized. Developing Language Implementations \u00b6 So far we have talked about what a language implementation is, but not yet how one is developed, which we will dive into now. Language implementations are by default fully generated from a high-level language specification using the Spoofax 3 compiler, thereby supporting iterative language development with low boilerplate. However, it is possible implement parts of or even the entire language/adapter project by hand, facilitating the integration of existing tools and languages. TODO: dynamic loading","title":"Anatomy of a language implementation"},{"location":"reference/anatomy_language_implementation/#anatomy-of-a-language-implementation","text":"In this section we give a high-level overview of what a Spoofax 3 language implementation is, dive into details, and explain how such a implementation can be manually written or completely generated from a high-level language specification","title":"Anatomy of a language implementation"},{"location":"reference/anatomy_language_implementation/#overview","text":"In essence, a language implementation in Spoofax 3 is nothing more than a standard Java library (e.g., a JAR file) with Java classes implementing or delegating to the various functionalities of the language such as parsing and transformations, as well as bundled resources such as a parse table which is loaded and interpreted at runtime. Therefore, Spoofax 3 language implementations are very easy to use in the Java ecosystem by just distributing the JAR file of the language, or by publishing/consuming it as a library with a build system such as Gradle. Furthermore, since no classloading or class generation is used, GraalVM native image can be used to ahead-of-time compile your language implementation into native code which does not require a JVM at all, and significantly reduces the startup time of your language. Diving deeper, a language implementation is actually split into three parts: a language project that contains the base functionality of the language, an adapter project that adapts the language project to the interface of Spoofax, and platform projects that plug the adapter project into various other platforms such as a command-line interface (CLI) and Eclipse plugin (TODO: more details on supported platforms in a separate section). We will first explain these projects and why this separation was chosen. TODO: diagram?","title":"Overview"},{"location":"reference/anatomy_language_implementation/#language-project","text":"A language project contains the base functionality of a language, such as a parser, syntax highlighter, analyzer, and compiler for the language. Such a project is unstructured : it does not have to adhere to any interface or data format. Therefore, it may use any tooling, libraries, and data structures to implement the base functionality. This facilitates integration of existing tools and minimal dependencies. A language project is just a Java library and can thus be used in a standalone fashion. However, there is no glue between base functionality, requiring manual implementation of a parse-analyze-compile pipeline for example. Furthermore, because the project is unstructured, we cannot provide any integration with other platforms such as a CLI and Eclipse plugin. Therefore, using a language project as a standalone library is a bit of a niche use case for when minimal dependencies or full control is absolutely necessary. Because it is such a niche use case, the default in Spoofax 3 is to merge it together with the adapter project. In essence, an adapter project adapts a language project to Spoofax 3. To understand why, we first explain the high-level architecture of Spoofax 3.","title":"Language Project"},{"location":"reference/anatomy_language_implementation/#spoofax-3-architecture-overview","text":"Spoofax 3 provides a general interface for language implementations: LanguageInstance , which is used by platforms to automatically plug languages into their platform. For example, LanguageInstance has functionality for syntax highlighting , which when given a resource of the language, returns a syntax highlighting for that resource. (TODO: more details on the functionality in LanguageInstance in a separate section) Furthermore, Spoofax 3 uses PIE ; a framework for building incremental pipelines, build systems, and compilers; to incrementalize the language implementation. Instead of directly computing the syntax highlighting for a resource, we create a task that returns the syntax highlighting when demanded, with PIE taking care of whether it should recompute the syntax highlighting because the resource (or the syntax highlighting implementation) changed, or if it can just be returned from a cache. (TODO: more details on PIE in a separate section) A platform such as Eclipse or IntelliJ can then take a LanguageInstance implementation, demand the syntax highlighting task, and show the result it in the editor for your language. Therefore, any language that implements LanguageInstance can get syntax highlighting in Eclipse, IntelliJ, and any other supported platforms for free, with PIE taking care of coarse-grained incrementalization. To receive the benefits of Spoofax 3, the adapter project must thus be implemented for your language.","title":"Spoofax 3 architecture overview"},{"location":"reference/anatomy_language_implementation/#adapter-project","text":"An adapter project implements Spoofax 3's LanguageInstance using the language project. This requires glue code between the unstructured language project and the structured LanguageInstance interface. For example, you would need to convert the data structure that the syntax highlighter of your language returns, to one that Spoofax 3 understands: the Styling class. Furthermore, because Spoofax 3 uses PIE, we also need to implement a PIE task definition that implements the (re)computing of syntax highlighting, as well as mark all dependencies that should cause the syntax highlighting to be recomputed. We also need to be able to instantiate your implementation of LanguageInstance . In case this is non-trivial, the recommended practice is to use dependency injection to achieve proper separation of concerns. A dependency injection framework such as Dagger is recommended (we use it extensively in Spoofax 3) because it catches dependency injection errors at compile-time, and does not require runtime class loading or generation. This may sound like you would need to write a lot of boilerplate. However, we provide a compiler that generates all this boilerplate for you. It is only necessary to write this boilerplate if you are integrating existing tooling. Even then, the compiler can generate some of the boilerplate for you. More details on the compiler can be found in the developing language implementations section.","title":"Adapter Project"},{"location":"reference/anatomy_language_implementation/#platform-projects","text":"TODO: every language-platform combination is a separate project to support ahead-of-time compilation, static loading, and customization of the platform project. CLI: can be ahead-of-time compiled with GraalVM native image to create a native Windows/macOS/Linux CLI for your language Eclipse/IntelliJ: statically loaded plugin that can be deployed with Eclipse/IntelliJ, and can be fully customized.","title":"Platform Projects"},{"location":"reference/anatomy_language_implementation/#developing-language-implementations","text":"So far we have talked about what a language implementation is, but not yet how one is developed, which we will dive into now. Language implementations are by default fully generated from a high-level language specification using the Spoofax 3 compiler, thereby supporting iterative language development with low boilerplate. However, it is possible implement parts of or even the entire language/adapter project by hand, facilitating the integration of existing tools and languages. TODO: dynamic loading","title":"Developing Language Implementations"},{"location":"reference/configuration/","text":"Configuration \u00b6 The main entry point of a language definition is the spoofaxc.cfg (Spoofax compiler configuration) file, written in the CFG language. The goal of this config file is to configure basic options, enable/disable features, point to main source files of meta-languages, to add/override behaviour, and to serve as an anchor on the filesystem. The directory that the spoofaxc.cfg file is in is called the \"root directory\" of the language definition, and any relative paths are resolved relative to that directory. The CFG language has domain-specific syntax for configuring language definitions. However, the syntax follows these conventions: Options are assigned a value with $Option = $Expression . Unless specified otherwise, options may only be given once. Sections $Section { ... } may enable features and group options. Lists $List [ ..., ... ] indicate an option/section may be given 0-many times. Let bindings let $Name = $Expression can be used to give values a name that can be (re-)used in the rest of the configuration file. If something in the documentation is unclear, the CFG language definition can be found here . In its most basic form, the spoofaxc.cfg file for a language named Calc looks as follows: name = \"Calc\" which assigns the string \"Calc\" to the name option. A more interesting example configures more options and enables syntax definition: name = \"Calc\" java-class-id-prefix = java Calc file-extension = \"calc\" sdf3 {} parser { default-start-symbol = sort Program } Here, java Calc is assigned to the java-class-id-prefix option. The sdf3 {} section from the example is empty, but is used to enable the SDF3 meta-language. The parser section enables generation of a parser, and also sets the default start symbol to use to sort Program . Literals \u00b6 Literals are expressions that are usually directly assigned to options, or bound to a name with let bindings. CFG has the following literals: Syntax Example(s) Type (true|false) true false Boolean -?[0-9]+ 1 -20 Integer [0-9]+ 1 20 Unsigned integer '(~[\\'\\$\\n\\r\\\\] | \\\\~[\\n\\r])' '[' 'l' Character \"(~[\\\"\\$\\n\\r\\\\] | \\\\~[\\n\\r])*\" \"foo\" \"bar\" String (./|/)~[\\n\\r\\,\\;\\]\\)\\}\\ ]* ./relative/file /absolute/file Filesystem path $JavaIdChars Java foo Java identifier $JavaQIdLit Java foo.bar.Baz Qualified Java identifier task-def $JavaQIdLit task-def foo.bar.Baz Qualified Java identifier that represents a task definition sort [a-zA-Z0-9\\-\\_\\$]+ sort Start SDF3 sort identifier strategy [a-zA-Z0-9\\-\\_\\$]+ strategy Start Stratego strategy identifier $CoordinateChars:$CoordinateChars:$CoordinateChars org.metaborg:strategolib:1.0.0 Coordinate $CoordinateChars:$CoordinateChars:$CoordinateRequirementChars org.metaborg:strategolib:* Coordinate requirement With the following syntax non-terminals: Name Syntax JavaIdChars [a-zA-Z\\_\\$][a-zA-Z0-9\\_\\$]* JavaQIdLit $JavaIdChars(\\.$JavaIdChars)* CoordinateChars (~[\\\"\\:\\;\\,\\*\\$\\{\\}\\[\\]\\n\\r\\\\\\ ]|\\\\~[\\n\\r]) CoordinateRequirementChars ($CoordinateChars|\\*) For Java, SDF3 sort, and Stratego strategy identifiers, the corresponding keywords of those languages are rejected as identifiers. Let bindings \u00b6 Let bindings of the form let $Name = $Expression bind a name to an expression, for example: let showParsedAst = task-def mb.helloworld.task.HelloWorldShowParsedAst let showParsedAstCommand = command-def { task-def = showParsedAst ... } editor-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand ... } ] ] creates a binding from name showParsedAst to task-def mb.helloworld.task.HelloWorldShowParsedAst , which we then pass to the task-def option in command-def . The command in turn is bound to showParsedAstCommand , assigned to the command-def option in a command-action section. Top-level options \u00b6 The following top-level options exist: Syntax Required? Description Type group = $Expression no Group identifier of the language, used as the group / groupId in the Java ecosystem. Defaults to org.metaborg . String id = $Expression no Artifact identifier of the language, used as the name / artifactId in the Java ecosystem. Defaults to the name of the language uncapitalized. String name = $Expression yes Name of the language. String version = $Expression no Version of the language, used as the version in the Java ecosystem. Defaults to 0.1.0 . String file-extension = $Expression no File extension of the language. May be given multiple times. Defaults to the name of the language transformed to fit in 3 characters. String java-package-id-prefix = $Expression no The prefix to add before all package identifiers in Java source files. Defaults to mb.$Name where $Name is transformed to be a valid package identifier. Qualified Java identifier java-class-id-prefix = $Expression no The prefix to add before all Java classes. Defaults to the name of the language transformed to be a valid class identifier. Java identifier source-directory = $Expression no Path relative to the root directory that has the sources of the language definition. Defaults to src . Path build-directory = $Expression no Path relative to the root directory that has the generated sources and build outputs when building the language definition. Defaults to build . Path Commands \u00b6 Commands are sections that are also expressions, typically assigned to a name with a let binding, with the following form: let $Name = command-def { $CommandOption* } The following options are available in a command: Syntax Required? Description Type task-def = $Expression yes The task definition that the command will execute. Qualified Java identifier that represents a task definition, or qualified Java identifier type = $Expression no The fully qualified Java type we want this command to be generated as. Can be omitted to generate a type based on the name of the task definition. Qualified Java identifier display-name = $Expression yes The display name of the command. String description = $Expression no The optional description of the command. String supported-execution-types = [($ExecutionType ,)*] no The optional supported execution types of the command. Defaults to [Once, Continuous] . n/a args-type = $Expression no The fully qualified Java type of the argument class. Can be omitted if the argument class is a nested class named Args of the task definition. Qualified Java identifier parameters = [ $Parameter* ] yes The description of the parameters of the command n/a The following ExecutionType s are supported: Once indicates a that this command supports being executed as a one-shot command. Continuous indicates that this command supports being executed every time the source file changes. A $Parameter has the form $Identifier { $ParameterOptions } with the following options: Syntax Required? Description Type type = $Expression yes The fully qualified Java of the type of the parameter. This must match the type of the parameter inside the args-type of the command. Qualified Java identifier required = $Expression no Whether the parameter is required. Defaults to true . Boolean converter-type = $Expression no The argument converter for this parameter, which can convert a String value to the type of this parameter. Must implement the ArgConverter interface. Qualified Java identifier argument-providers = [($ArgumentProvider ,)*] no Argument providers for this parameter that attempt to automatically provide a fitting argument. When providing an argument fails, the next argument provider in the list will be attempted. If no arguments can be provided, and the argument is required, then the argument must be provided by the user that executes the command, or executing the command will fail. n/a The following ArgumentProvider s are supported: Value($Expression) provides a default value given by the expression. The expression must match the type of the parameter, even though this is not currently checked. Context($CommandContext) attempts to infer the argument by context. The following CommandContext s are supported: Directory : attempt to infer a ResourcePath to an existing directory. For example, when right-clicking a directory in an IDE to execute a command on that directory. File : attempt to infer a ResourcePath to an existing file. For example, when right-clicking a file in an IDE to execute a command on that directory, or when executing a command in an editor for a file. HierarchicalResource : attempt to infer a ResourcePath to a hierarchical resource. A hierarchical resource is a resource that belongs to a (tree) hierarchy, such as a file or directory on the local filesystem. Use this when the command relies on the resource being in a filesystem, but does not care whether it is a directory or a file. ReadableResource : attempt to infer a ResourceKey to a readable resource. This is more general than File , as we only ask for a resource that can be read, not one that belongs to a (local) filesystem. Use this when the command does not rely on the resource being in a filesystem. Region : attempt to infer a Region in a source file. Inference succeeds when the context has a selection of size 1 or larger. For example, when executing a command in an editor that has a selection, the region will be that selection. Offset : attempt to infer an int representing an offset in a source file. Inference succeeds when the context has a cursor offset (i.e., a selection of size 0 or larger). For example, when executing a command in an editor, the offset will be the offset to the cursor in the editor. EnclosingContext($EnclosingCommandContext) attempts to infer the argument by the enclosing context. The following EnclosingCommandContext s are supported: Project : attempt to infer a ResourcePath to the enclosing project. For example, when executing a command in an IDE on a file, directory, or editor for a file, that belongs to a project. Or when executing a command in a CLI, the directory will be the current working directory. Directory : attempt to infer a ResourcePath to the enclosing directory. For example, when executing a command in the context of a file, directory, or editor for a file, the directory will be the parent of that file/directory. Here is an example of a command that shows the parsed AST by taking one file argument that is inferred from context: let showParsedAstCommand = command-def { type = java mb.helloworld.command.HelloWorldShowParsedAstCommand task-def = showParsedAst args-type = java mb.helloworld.task.HelloWorldShowParsedAst.Args display-name = \"Show parsed AST\" description = \"Shows the parsed AST\" supported-execution-types = [Once, Continuous] parameters = [ file = parameter { type = java mb.resource.ResourceKey required = true argument-providers = [Context(ResourceKey)] } ] } Menu items \u00b6 Menu items take the form of: a separator representing a horizontal line in a menu used to separate groups of menu items. a menu $Expression [ $MenuItem* ] representing a (nested) menu with a display name defined by the expression which must be a string, and a list of nested menu items. a command-action { $CommandActionOption* } representing an action that executes a command when a user clicks on it. A command action has the following options: Syntax Required? Description Type command-def = $Expression yes The command to execute. Command or qualified Java identifier execution-type = $ExecutionType yes How the command should be executed. n/a required-resource-types = [($ResourceType ,)*] no On which kinds of resources this menu item will be shown on resource context menus. Defaults to empty. If empty, it will not be hidden based on resources. n/a required-enclosing-resource-types = [($EnclosingResourceType ,)*] no On which kinds of enclosing resources this menu item will be shown on resource context menus. Defaults to empty. If empty, it will not be hidden based on enclosing resource. n/a required-editor-file-types = [($EditorFileType ,)*] no On which kinds of editors belonging to certain file types this menu item will be shown. Defaults to empty. If empty, it will not be hidden based on editor file types. n/a required-editor-selection-types = [($EditorSelectionType ,)*] no On which kinds of editor selection types this menu item will be shown. Defaults to empty. If empty, it will not be hidden based on editor selections. n/a display-name = $Expression no The display name of the command action. Defaults to the display name of the command String description = $Expression no The description of he command action. Defaults to the description of the command String The following ResourceType s are supported: Directory : the menu item will only be shown when a directory is selected. File : the menu item will only be shown when a file is selected. The following EnclosingResourceType s are supported: Project : the menu item will only be shown when the selected resource has an enclosing project. Directory: : the menu item will only be shown when the selected resource has an enclosing directory. The following EditorFileType s are supported: HierarchicalResource : the menu item will only be shown when the editor belongs to a hierarchical resource. That is, a resource that belongs to a tree, such as a file or directory on the local filesystem. ReadableResource : the menu item will only be shown when the editor belongs to a readable resource. The following EditorSelectionType s are supported: Region : the menu item will only be shown when a region with size >0 in the source file is selected. Offset : the menu item will only be shown in the context of an editor with a cursor. Menus \u00b6 Menu items are assigned to 3 particular menus: editor-context-menu [ $MenuItem* ] : the context menu that gets shown in editors of the language, for example when right-clicking in an editor of the language in an IDE. Spoofax automatically creates a top-level submenu with the name of the language to host the editor context menu items. The required-editor-file-types and required-editor-selection-types options are used to filter menu items. resource-context-menu [ $MenuItem* ] : the context menu that gets shown in resource explorers, for example when right-clicking in the file browser in an IDE. Spoofax automatically creates a top-level submenu with the name of the language to host the editor context menu items. The required-resource-types and required-enclosing-resource-types options are used to filter menu items. main-menu [ $MenuIitem* ] : the main menu of the language, which is shown on the menu bar in IDEs. When no main-menu section is given, it defaults to the same menu as editor-context-menu . For example, we can assign the command defined earlier to several menus: editor-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand execution-type = Once } command-action { command-def = showParsedAstCommand execution-type = Continuous } ] ] resource-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand execution-type = Once required-resource-types = [File] } ] ] Language feature sections \u00b6 Parser \u00b6 The parser { $ParserOption* } section enables generation of a parser, and groups options. The sdf3 section must be enabled when the parser section is enabled. The following ParserOption s are supported: Syntax Required? Description Type default-start-symbol = $Expression yes The start symbol to use when no specific one is provided. SDF3 sort identifier variant = $ParserVariant no The parser variant to use. Defaults to jsglr1 . n/a The following ParserVariant s are supported: jsglr1 : uses the JSGLR1 parser. jsglr2 { $Jsglr2Option* } : uses the JSGLR2 parser. The following Jsglr2Option s are supported: preset = $Jsglr2Preset : sets the JSGLR2 preset to use. The following Jsglr2Preset s are supported: Standard Elkhound Recovery RecoveryElkhound DataDependent LayoutSensitive Composite Incremental IncrementalRecovery Comment symbols \u00b6 The comment-symbols { $CommentSymbolOption* } section enables specification of line and block comment characters, which are required for the \"toggle comment\" editor service. The following CommentSymbolOption s are supported: Syntax Required? Description Type line = $Expression no Adds a line comment symbol. Can be given multiple times to list multiple line comment symbols. The first one will be used to comment a line with the \"toggle comment\" editor service. String block = $Expression * $Expression no Adds block comment symbols, with an opening and close symbol. Current \"toggle comment\" editor services do not use block comment symbols yet. String Bracket symbols \u00b6 The bracket-symbols { $BracketSymbolOption* } section enables specification of bracket symbols (e.g., square brackets, curly brackets, parentheses, etc.), which are required for the \"bracket matching\" editor service. The following BracketSymbolOption s are supported: Syntax Required? Description Type bracket = $Expression * $Expression no Adds bracket symbols, with an opening and closing symbol. Can be given multiple times to list multiple bracket symbols. Character Styler \u00b6 The styler { $StylerOption* } section enables generation of a styler, and groups options. The esv section must be enabled when the styler section is enabled. Currently, no StylerOption s are supported. Constraint analyzer \u00b6 The constraint-analyzer { $ConstraintAnalyzerOption* } section enables generation of a constraint analyzer, and groups options. The statix section must be enabled when the constraint-analyzer section is enabled. The following ConstraintAnalyzerOption s are supported: Syntax Required? Description Type multi-file = $Expression no Whether multi-file analysis is enabled. Defaults to false . Boolean stratego-strategy = $Expression no The stratego strategy entry-point that handles communication with the constraint-solver. Defaults to editor-analyze . Stratego strategy identifier default-statix-message-stacktrace-length = $Expression no The default Statix message stacktrace length to use. Default is implementation-defined. Does nothing if Statix is not enabled. Unsigned integer default-statix-message-term-depth = $Expression no The default Statix message term depth to use. Default is implementation-defined. Does nothing if Statix is not enabled. Unsigned integer default-statix-test-log-level = $Expression no The default Statix test log level to use. Default is implementation-defined. Does nothing if Statix is not enabled. String default-statix-supress-cascading-errors = $Expression no Whether to suppress cascading errors by default. Default is implementation-defined. Does nothing if Statix is not enabled. Boolean statix-solver-mode = $StatixSolverMode no Statix solver mode. Defaults to traditional . Does nothing if Statix is not enabled. n/a The following StatixSolverMode s are supported: traditional concurrent incremental Multi-language analyzer \u00b6 The multilang-analyzer { $MultilangAnalyzerOption* } section enables generation of a multi-language analyzer, and groups options. The constraint-analyzer and statix sections must be enabled when the multilang-analyzer section is enabled. Currently, no MultilangAnalyzerOption s are supported. Stratego runtime \u00b6 The stratego-runtime { $StrategoRuntimeOption* } section enables generation of a stratego runtime, and groups options. The stratego section must be enabled when the stratego-runtime section is enabled. The following StrategoRuntimeOption s are supported: Syntax Required? Description Type strategy-package-id = $Expression no Adds a package as a private Stratego package. Can be specified multiple times. Java package identifier interop-registerer-by-reflection = $Expression no Adds an interop registerer to load by reflection. Can be specified multiple times. Java type identifier add-spoofax2-primitives = $Expression no Whether to add the Spoofax 2 Stratego primitives. Boolean add-nabl2-primitives = $Expression no Whether to add the NaBL2 Stratego primitives. Boolean add-statix-primitives = $Expression no Whether to add the Statix Stratego primitives. Boolean with-primitive-library = $Expression no Adds a Stratego primitive strategies library (implementing org.spoofax.interpreter.library.IOperatorRegistry ) to the generated StrategoRuntimeBuilderFactory . The library must have an @Inject constructor. Can be specified multiple times. Java type identifier with-interop-registerer = $Expression no Adds a Stratego interop registerer (implementing org.strategoxt.lang.InteropRegisterer ) to the generated StrategoRuntimeBuilderFactory . The registerer must have an @Inject constructor. Can be specified multiple times. Java type identifier class-kind = $Expression no Specifies whether the classes are generated ( Generated ) or provided manually ( Manual ). Defaults to Generated . Generated or Manual base-StrategoRuntimeBuilderFactory = $Expression no Package and name of the generated StrategoRuntimeBuilderFactory . Java type identifier extend-StrategoRuntimeBuilderFactory = $Expression no Package and name of the extending StrategoRuntimeBuilderFactory , if any. Java type identifier Completer \u00b6 The completer { $CompleterOption* } section enables generation of a code completer, and groups options. The constraint-analyzer and statix sections must be enabled when the completer section is enabled. Currently, no CompleterOption s are supported. Reference resolution \u00b6 The reference-resolution { $ReferenceResolutionOption* } section enables generation of the reference resolver editor service, and groups options. The following ReferenceResolutionOption s are supported: Syntax Required? Description Type variant = $ReferenceResolutionVariant yes The reference resolution variant to use. n/a The following ReferenceResolutionVariant s are supported: Stratego-based: stratego { strategy = strategy $Strategy } where Strategy is a Stratego strategy, typically editor-resolve . Hover tooltips \u00b6 The hover { $HoverOption* } section enables generation of the hover text editor service, and groups options. The following HoverOption s are supported: Syntax Required? Description Type variant = $HoverVariant yes The reference resolution variant to use. n/a The following HoverVariant s are supported: Stratego-based: stratego { strategy = strategy $Strategy } where Strategy is a Stratego strategy, typically editor-hover . Meta-language sections \u00b6 SDF3 \u00b6 The sdf3 { $Sdf3Option* } section enables syntax definition with SDF3 . The parser section must be enabled when the sdf3 section is enabled. The following Sdf3Option s are supported: Syntax Required? Description Type source = $Sdf3Source no The source of the SDF3 definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./start.sdf3 as its main file relative to the main source directory. n/a The following $Sdf3Source s are supported: Files: files { $Sdf3FilesOption* } Prebuilt: prebuilt { $Sdf3PrebuiltOption } The following Sdf3FilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main SDF3 file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main SDF3 file relative to the main-source-directory . Defaults to ./start.sdf3 . Path include-directory = $Expression no Adds an include directory from which to resolve SDF3 imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the SDF3 files in it accessable to dependencies. May be given multiple times. Path parse-table-generator { $ParseTableGeneratorOption* } no Parse table generator options. n/a placeholders { $PlaceholderOption* } no Placeholder syntax used in code completion. n/a stratego-concrete-syntax-extension-main-file = $Expression no Sets the main SDF3 file used to create a concrete syntax extension parse table. Path The following $Sdf3PrebuiltOption s are supported: Syntax Required? Description Type parse-table-aterm-file = $Expression yes The prebuilt SDF3 parse table ATerm file to use (usually called sdf.tbl ) relative to the root directory Path parse-table-persisted-file = $Expression yes The prebuilt SDF3 parse table persisted file to use (usually called sdf.bin ) relative to the root directory Path The following ParseTableGeneratorOption s are supported: Syntax Required? Description Type dynamic = $Expression no Whether the generated parse table is dynamic. Defaults to false . Boolean data-dependent = $Expression no Whether the generated parse table is data-dependent. Defaults to false . Boolean layout-sensitive = $Expression no Whether the generated parse table is layout-sensitive. Defaults to false . Boolean solve-deep-conflicts = $Expression no Whether the parse table generator solves deep priority conflicts. Defaults to true . Boolean check-overlap = $Expression no Whether the parse table generator checks for overlap. Defaults to false . Boolean check-priorities = $Expression no Whether the parse table generator checks priorities. Defaults to false . Boolean The following PlaceholderOption s are supported: Syntax Required? Description Type prefix = $Expression no The prefix to a placholder. Defaults to [[ , or empty if the other is defined. String postfix = $Expression no The postfix to a placeholder. Defaults to ]] , or empty if the other is defined. String ESV \u00b6 The esv { $EsvOption* } section enables syntax-based styling definition with ESV . The styler section must be enabled when the esv section is enabled. The following EsvOption s are supported: Syntax Required? Description Type source = $EsvSource no The source of the ESV definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.esv as its main file relative to the main source directory. n/a The following $EsvSource s are supported: Files: files { $EsvFilesOption* } Prebuilt: prebuilt { $EsvPrebuiltOption } The following EsvFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main ESV file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main ESV file relative to the main-source-directory . Defaults to ./main.esv . Path include-directory = $Expression no Adds an include directory from which to resolve ESV imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the ESV files in it accessable to dependencies. May be given multiple times. Path The following $EsvPrebuiltOption s are supported: Syntax Required? Description Type file = $Expression yes The prebuilt ESV file to use relative to the root directory Path Statix \u00b6 The statix { $StatixOption* } section enables static semantics definition with Statix . The constraint-anaylzer section must be enabled when the statix section is enabled. The following StatixOption s are supported: Syntax Required? Description Type source = $StatixSource no The source of the Statix definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.stx as its main file relative to the main source directory. n/a The following $StatixSource s are supported: Files: files { $StatixFilesOption* } Prebuilt: prebuilt { $StatixPrebuiltOption } The following StatixFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main Statix file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main Statix file relative to the main-source-directory . Defaults to ./main.stx . Path include-directory = $Expression no Adds an include directory from which to resolve Statix imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the Statix files in it accessable to dependencies. May be given multiple times. Path sdf3-statix-signature-generation = $Expression no Whether SDF3 to Statix signature generation is enabled. When enabled, stratego { sdf3-statix-explication-generation = true } must also be enabled. Defaults to false . Boolean The following $StatixPrebuiltOption s are supported: Syntax Required? Description Type spec-aterm-directory = $Expression yes The prebuilt Statix spec ATerm directory to use relative to the root directory Path Stratego \u00b6 The stratego { $StrategoOption* } section enables definition of transformations with Stratego . The stratego-runtime section must be enabled when the stratego section is enabled. The following StrategoOption s are supported: Syntax Required? Description Type source = $StrategoSource no The source of the Statix definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.str2 as its main file relative to the main source directory. n/a output-java-package = $Expression no The Java package into which compiled Stratego Java files are generated. Defaults to the language's package, followed by .strategies . String The following $StrategoSource s are supported: Files: files { $StrategoFilesOption* } The following $StrategoFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main Stratego file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main Stratego file relative to the main-source-directory . Defaults to ./main.str2 . Path include-directory = $Expression no Adds an include directory from which to resolve Stratego imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the Stratego files in it accessable to dependencies. May be given multiple times. Path with-import-strategy-package-id = $Expression no Adds a Java package to import in the Java files that have been compiled from Stratego files. For example, \"mb.mylang.strategies\" adds import import mb.mylang.strategies.* . May be given multiple times. String sdf3-statix-explication-generation = $Expression no Whether SDF3 to Statix injection explication/implication generation is enabled. When enabled, statix { sdf3-statix-signature-generation = true } must also be enabled. Defaults to false . Boolean language-strategy-affix = $Expression no The affix that is used to make certain generated strategies unique to the language. This is used both as a prefix and suffix. Defaults to name of the language transformed to a Stratego strategy identifier. Stratego strategy identifier concrete-syntax-extension-parse-table = $Expression no Adds a Stratego concrete syntax extension parse table that can be used to parse Stratego files with. May be given multiple times. Path Dependencies \u00b6 The dependencies [ $Dependency* ] section allows specifying dependencies to other language (library) projects. A dependency consists of a dependency expression ( $Exp ) that specifies the source of the dependency. The following expressions are supported: $Coordinate : an exact coordinate to a project, such as org.metaborg:strategolib:1.0.0 . Coordinates are resolved to a statically loaded language, or a dynamically loaded language definition. $CoordinateRequirement : a coordinate to a project with an open version, such as org.metaborg:strategolib:* , resolved identically to $Coordinate except that the latest available version will be chosen. $Path : a relative path (relative to the directory spoofaxc.cfg is in) to a language definition. Dependencies can be of a certain kind ( $DependencyKind ): Build : a dependency that is resolved at build time , meaning when the language definition is built, allowing the use of sources and binaries of the project the dependency points to. Run : a dependency that is resolved at run time , meaning when the language is executed, allowing use of the classes and tasks of the project the dependency points to. Note: this kind of dependency has not yet been implemented. The following Dependency s are supported: Syntax Description $Exp A dependency to a project defined by the expression, available at Build and Run time. $Exp { $DependencyOption* } A dependency to project defined by the expression, with configuration options. The following DependencyOption s are supported: kinds = [ $DependencyKind* ] : set the kinds at which this dependency is resolved. Additionally, build time dependencies can be quickly defined with the build-dependencies [ $BuildDependency* ] section. The following BuildDependency s are supported: Syntax Description $Exp A dependency to project defined by the expression, available only at Build time. For example, the following configuration adds build dependencies to common libraries shipped with Spoofax 3: build-dependencies [ org.metaborg : strategolib:* org.metaborg : gpp:* org.metaborg : libspoofax2:* org.metaborg : libstatix:* ] spoofaxc.lock \u00b6 The spoofaxc.lock file, which resides next to the spoofaxc.cfg file, contains values for several options that have defaults derived from other options, in order to keep these derived values stable even when the options they are derived from are changed. For example, when no java-class-id-prefix option is set in spoofaxc.cfg , it will be derived from the name option with some changes to make it compatible as a Java identifier, and is stored under shared.defaultClassPrefix in the spoofaxc.lock file. When you change the name of your language, the stored value will be used, keeping the class prefix the same, making it possible to rename the language without having to rename all class files. Therefore, the spoofaxc.lock file should be checked in to source control, in order to have reproducible builds. If you do want to re-derive a default from other options, remove the option from the spoofaxc.lock file and rebuild the language. The value will be re-derived and stored in spoofaxc.lock , after which you need to check it into source control again.","title":"Configuration"},{"location":"reference/configuration/#configuration","text":"The main entry point of a language definition is the spoofaxc.cfg (Spoofax compiler configuration) file, written in the CFG language. The goal of this config file is to configure basic options, enable/disable features, point to main source files of meta-languages, to add/override behaviour, and to serve as an anchor on the filesystem. The directory that the spoofaxc.cfg file is in is called the \"root directory\" of the language definition, and any relative paths are resolved relative to that directory. The CFG language has domain-specific syntax for configuring language definitions. However, the syntax follows these conventions: Options are assigned a value with $Option = $Expression . Unless specified otherwise, options may only be given once. Sections $Section { ... } may enable features and group options. Lists $List [ ..., ... ] indicate an option/section may be given 0-many times. Let bindings let $Name = $Expression can be used to give values a name that can be (re-)used in the rest of the configuration file. If something in the documentation is unclear, the CFG language definition can be found here . In its most basic form, the spoofaxc.cfg file for a language named Calc looks as follows: name = \"Calc\" which assigns the string \"Calc\" to the name option. A more interesting example configures more options and enables syntax definition: name = \"Calc\" java-class-id-prefix = java Calc file-extension = \"calc\" sdf3 {} parser { default-start-symbol = sort Program } Here, java Calc is assigned to the java-class-id-prefix option. The sdf3 {} section from the example is empty, but is used to enable the SDF3 meta-language. The parser section enables generation of a parser, and also sets the default start symbol to use to sort Program .","title":"Configuration"},{"location":"reference/configuration/#literals","text":"Literals are expressions that are usually directly assigned to options, or bound to a name with let bindings. CFG has the following literals: Syntax Example(s) Type (true|false) true false Boolean -?[0-9]+ 1 -20 Integer [0-9]+ 1 20 Unsigned integer '(~[\\'\\$\\n\\r\\\\] | \\\\~[\\n\\r])' '[' 'l' Character \"(~[\\\"\\$\\n\\r\\\\] | \\\\~[\\n\\r])*\" \"foo\" \"bar\" String (./|/)~[\\n\\r\\,\\;\\]\\)\\}\\ ]* ./relative/file /absolute/file Filesystem path $JavaIdChars Java foo Java identifier $JavaQIdLit Java foo.bar.Baz Qualified Java identifier task-def $JavaQIdLit task-def foo.bar.Baz Qualified Java identifier that represents a task definition sort [a-zA-Z0-9\\-\\_\\$]+ sort Start SDF3 sort identifier strategy [a-zA-Z0-9\\-\\_\\$]+ strategy Start Stratego strategy identifier $CoordinateChars:$CoordinateChars:$CoordinateChars org.metaborg:strategolib:1.0.0 Coordinate $CoordinateChars:$CoordinateChars:$CoordinateRequirementChars org.metaborg:strategolib:* Coordinate requirement With the following syntax non-terminals: Name Syntax JavaIdChars [a-zA-Z\\_\\$][a-zA-Z0-9\\_\\$]* JavaQIdLit $JavaIdChars(\\.$JavaIdChars)* CoordinateChars (~[\\\"\\:\\;\\,\\*\\$\\{\\}\\[\\]\\n\\r\\\\\\ ]|\\\\~[\\n\\r]) CoordinateRequirementChars ($CoordinateChars|\\*) For Java, SDF3 sort, and Stratego strategy identifiers, the corresponding keywords of those languages are rejected as identifiers.","title":"Literals"},{"location":"reference/configuration/#let-bindings","text":"Let bindings of the form let $Name = $Expression bind a name to an expression, for example: let showParsedAst = task-def mb.helloworld.task.HelloWorldShowParsedAst let showParsedAstCommand = command-def { task-def = showParsedAst ... } editor-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand ... } ] ] creates a binding from name showParsedAst to task-def mb.helloworld.task.HelloWorldShowParsedAst , which we then pass to the task-def option in command-def . The command in turn is bound to showParsedAstCommand , assigned to the command-def option in a command-action section.","title":"Let bindings"},{"location":"reference/configuration/#top-level-options","text":"The following top-level options exist: Syntax Required? Description Type group = $Expression no Group identifier of the language, used as the group / groupId in the Java ecosystem. Defaults to org.metaborg . String id = $Expression no Artifact identifier of the language, used as the name / artifactId in the Java ecosystem. Defaults to the name of the language uncapitalized. String name = $Expression yes Name of the language. String version = $Expression no Version of the language, used as the version in the Java ecosystem. Defaults to 0.1.0 . String file-extension = $Expression no File extension of the language. May be given multiple times. Defaults to the name of the language transformed to fit in 3 characters. String java-package-id-prefix = $Expression no The prefix to add before all package identifiers in Java source files. Defaults to mb.$Name where $Name is transformed to be a valid package identifier. Qualified Java identifier java-class-id-prefix = $Expression no The prefix to add before all Java classes. Defaults to the name of the language transformed to be a valid class identifier. Java identifier source-directory = $Expression no Path relative to the root directory that has the sources of the language definition. Defaults to src . Path build-directory = $Expression no Path relative to the root directory that has the generated sources and build outputs when building the language definition. Defaults to build . Path","title":"Top-level options"},{"location":"reference/configuration/#commands","text":"Commands are sections that are also expressions, typically assigned to a name with a let binding, with the following form: let $Name = command-def { $CommandOption* } The following options are available in a command: Syntax Required? Description Type task-def = $Expression yes The task definition that the command will execute. Qualified Java identifier that represents a task definition, or qualified Java identifier type = $Expression no The fully qualified Java type we want this command to be generated as. Can be omitted to generate a type based on the name of the task definition. Qualified Java identifier display-name = $Expression yes The display name of the command. String description = $Expression no The optional description of the command. String supported-execution-types = [($ExecutionType ,)*] no The optional supported execution types of the command. Defaults to [Once, Continuous] . n/a args-type = $Expression no The fully qualified Java type of the argument class. Can be omitted if the argument class is a nested class named Args of the task definition. Qualified Java identifier parameters = [ $Parameter* ] yes The description of the parameters of the command n/a The following ExecutionType s are supported: Once indicates a that this command supports being executed as a one-shot command. Continuous indicates that this command supports being executed every time the source file changes. A $Parameter has the form $Identifier { $ParameterOptions } with the following options: Syntax Required? Description Type type = $Expression yes The fully qualified Java of the type of the parameter. This must match the type of the parameter inside the args-type of the command. Qualified Java identifier required = $Expression no Whether the parameter is required. Defaults to true . Boolean converter-type = $Expression no The argument converter for this parameter, which can convert a String value to the type of this parameter. Must implement the ArgConverter interface. Qualified Java identifier argument-providers = [($ArgumentProvider ,)*] no Argument providers for this parameter that attempt to automatically provide a fitting argument. When providing an argument fails, the next argument provider in the list will be attempted. If no arguments can be provided, and the argument is required, then the argument must be provided by the user that executes the command, or executing the command will fail. n/a The following ArgumentProvider s are supported: Value($Expression) provides a default value given by the expression. The expression must match the type of the parameter, even though this is not currently checked. Context($CommandContext) attempts to infer the argument by context. The following CommandContext s are supported: Directory : attempt to infer a ResourcePath to an existing directory. For example, when right-clicking a directory in an IDE to execute a command on that directory. File : attempt to infer a ResourcePath to an existing file. For example, when right-clicking a file in an IDE to execute a command on that directory, or when executing a command in an editor for a file. HierarchicalResource : attempt to infer a ResourcePath to a hierarchical resource. A hierarchical resource is a resource that belongs to a (tree) hierarchy, such as a file or directory on the local filesystem. Use this when the command relies on the resource being in a filesystem, but does not care whether it is a directory or a file. ReadableResource : attempt to infer a ResourceKey to a readable resource. This is more general than File , as we only ask for a resource that can be read, not one that belongs to a (local) filesystem. Use this when the command does not rely on the resource being in a filesystem. Region : attempt to infer a Region in a source file. Inference succeeds when the context has a selection of size 1 or larger. For example, when executing a command in an editor that has a selection, the region will be that selection. Offset : attempt to infer an int representing an offset in a source file. Inference succeeds when the context has a cursor offset (i.e., a selection of size 0 or larger). For example, when executing a command in an editor, the offset will be the offset to the cursor in the editor. EnclosingContext($EnclosingCommandContext) attempts to infer the argument by the enclosing context. The following EnclosingCommandContext s are supported: Project : attempt to infer a ResourcePath to the enclosing project. For example, when executing a command in an IDE on a file, directory, or editor for a file, that belongs to a project. Or when executing a command in a CLI, the directory will be the current working directory. Directory : attempt to infer a ResourcePath to the enclosing directory. For example, when executing a command in the context of a file, directory, or editor for a file, the directory will be the parent of that file/directory. Here is an example of a command that shows the parsed AST by taking one file argument that is inferred from context: let showParsedAstCommand = command-def { type = java mb.helloworld.command.HelloWorldShowParsedAstCommand task-def = showParsedAst args-type = java mb.helloworld.task.HelloWorldShowParsedAst.Args display-name = \"Show parsed AST\" description = \"Shows the parsed AST\" supported-execution-types = [Once, Continuous] parameters = [ file = parameter { type = java mb.resource.ResourceKey required = true argument-providers = [Context(ResourceKey)] } ] }","title":"Commands"},{"location":"reference/configuration/#menu-items","text":"Menu items take the form of: a separator representing a horizontal line in a menu used to separate groups of menu items. a menu $Expression [ $MenuItem* ] representing a (nested) menu with a display name defined by the expression which must be a string, and a list of nested menu items. a command-action { $CommandActionOption* } representing an action that executes a command when a user clicks on it. A command action has the following options: Syntax Required? Description Type command-def = $Expression yes The command to execute. Command or qualified Java identifier execution-type = $ExecutionType yes How the command should be executed. n/a required-resource-types = [($ResourceType ,)*] no On which kinds of resources this menu item will be shown on resource context menus. Defaults to empty. If empty, it will not be hidden based on resources. n/a required-enclosing-resource-types = [($EnclosingResourceType ,)*] no On which kinds of enclosing resources this menu item will be shown on resource context menus. Defaults to empty. If empty, it will not be hidden based on enclosing resource. n/a required-editor-file-types = [($EditorFileType ,)*] no On which kinds of editors belonging to certain file types this menu item will be shown. Defaults to empty. If empty, it will not be hidden based on editor file types. n/a required-editor-selection-types = [($EditorSelectionType ,)*] no On which kinds of editor selection types this menu item will be shown. Defaults to empty. If empty, it will not be hidden based on editor selections. n/a display-name = $Expression no The display name of the command action. Defaults to the display name of the command String description = $Expression no The description of he command action. Defaults to the description of the command String The following ResourceType s are supported: Directory : the menu item will only be shown when a directory is selected. File : the menu item will only be shown when a file is selected. The following EnclosingResourceType s are supported: Project : the menu item will only be shown when the selected resource has an enclosing project. Directory: : the menu item will only be shown when the selected resource has an enclosing directory. The following EditorFileType s are supported: HierarchicalResource : the menu item will only be shown when the editor belongs to a hierarchical resource. That is, a resource that belongs to a tree, such as a file or directory on the local filesystem. ReadableResource : the menu item will only be shown when the editor belongs to a readable resource. The following EditorSelectionType s are supported: Region : the menu item will only be shown when a region with size >0 in the source file is selected. Offset : the menu item will only be shown in the context of an editor with a cursor.","title":"Menu items"},{"location":"reference/configuration/#menus","text":"Menu items are assigned to 3 particular menus: editor-context-menu [ $MenuItem* ] : the context menu that gets shown in editors of the language, for example when right-clicking in an editor of the language in an IDE. Spoofax automatically creates a top-level submenu with the name of the language to host the editor context menu items. The required-editor-file-types and required-editor-selection-types options are used to filter menu items. resource-context-menu [ $MenuItem* ] : the context menu that gets shown in resource explorers, for example when right-clicking in the file browser in an IDE. Spoofax automatically creates a top-level submenu with the name of the language to host the editor context menu items. The required-resource-types and required-enclosing-resource-types options are used to filter menu items. main-menu [ $MenuIitem* ] : the main menu of the language, which is shown on the menu bar in IDEs. When no main-menu section is given, it defaults to the same menu as editor-context-menu . For example, we can assign the command defined earlier to several menus: editor-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand execution-type = Once } command-action { command-def = showParsedAstCommand execution-type = Continuous } ] ] resource-context-menu [ menu \"Debug\" [ command-action { command-def = showParsedAstCommand execution-type = Once required-resource-types = [File] } ] ]","title":"Menus"},{"location":"reference/configuration/#language-feature-sections","text":"","title":"Language feature sections"},{"location":"reference/configuration/#parser","text":"The parser { $ParserOption* } section enables generation of a parser, and groups options. The sdf3 section must be enabled when the parser section is enabled. The following ParserOption s are supported: Syntax Required? Description Type default-start-symbol = $Expression yes The start symbol to use when no specific one is provided. SDF3 sort identifier variant = $ParserVariant no The parser variant to use. Defaults to jsglr1 . n/a The following ParserVariant s are supported: jsglr1 : uses the JSGLR1 parser. jsglr2 { $Jsglr2Option* } : uses the JSGLR2 parser. The following Jsglr2Option s are supported: preset = $Jsglr2Preset : sets the JSGLR2 preset to use. The following Jsglr2Preset s are supported: Standard Elkhound Recovery RecoveryElkhound DataDependent LayoutSensitive Composite Incremental IncrementalRecovery","title":"Parser"},{"location":"reference/configuration/#comment-symbols","text":"The comment-symbols { $CommentSymbolOption* } section enables specification of line and block comment characters, which are required for the \"toggle comment\" editor service. The following CommentSymbolOption s are supported: Syntax Required? Description Type line = $Expression no Adds a line comment symbol. Can be given multiple times to list multiple line comment symbols. The first one will be used to comment a line with the \"toggle comment\" editor service. String block = $Expression * $Expression no Adds block comment symbols, with an opening and close symbol. Current \"toggle comment\" editor services do not use block comment symbols yet. String","title":"Comment symbols"},{"location":"reference/configuration/#bracket-symbols","text":"The bracket-symbols { $BracketSymbolOption* } section enables specification of bracket symbols (e.g., square brackets, curly brackets, parentheses, etc.), which are required for the \"bracket matching\" editor service. The following BracketSymbolOption s are supported: Syntax Required? Description Type bracket = $Expression * $Expression no Adds bracket symbols, with an opening and closing symbol. Can be given multiple times to list multiple bracket symbols. Character","title":"Bracket symbols"},{"location":"reference/configuration/#styler","text":"The styler { $StylerOption* } section enables generation of a styler, and groups options. The esv section must be enabled when the styler section is enabled. Currently, no StylerOption s are supported.","title":"Styler"},{"location":"reference/configuration/#constraint-analyzer","text":"The constraint-analyzer { $ConstraintAnalyzerOption* } section enables generation of a constraint analyzer, and groups options. The statix section must be enabled when the constraint-analyzer section is enabled. The following ConstraintAnalyzerOption s are supported: Syntax Required? Description Type multi-file = $Expression no Whether multi-file analysis is enabled. Defaults to false . Boolean stratego-strategy = $Expression no The stratego strategy entry-point that handles communication with the constraint-solver. Defaults to editor-analyze . Stratego strategy identifier default-statix-message-stacktrace-length = $Expression no The default Statix message stacktrace length to use. Default is implementation-defined. Does nothing if Statix is not enabled. Unsigned integer default-statix-message-term-depth = $Expression no The default Statix message term depth to use. Default is implementation-defined. Does nothing if Statix is not enabled. Unsigned integer default-statix-test-log-level = $Expression no The default Statix test log level to use. Default is implementation-defined. Does nothing if Statix is not enabled. String default-statix-supress-cascading-errors = $Expression no Whether to suppress cascading errors by default. Default is implementation-defined. Does nothing if Statix is not enabled. Boolean statix-solver-mode = $StatixSolverMode no Statix solver mode. Defaults to traditional . Does nothing if Statix is not enabled. n/a The following StatixSolverMode s are supported: traditional concurrent incremental","title":"Constraint analyzer"},{"location":"reference/configuration/#multi-language-analyzer","text":"The multilang-analyzer { $MultilangAnalyzerOption* } section enables generation of a multi-language analyzer, and groups options. The constraint-analyzer and statix sections must be enabled when the multilang-analyzer section is enabled. Currently, no MultilangAnalyzerOption s are supported.","title":"Multi-language analyzer"},{"location":"reference/configuration/#stratego-runtime","text":"The stratego-runtime { $StrategoRuntimeOption* } section enables generation of a stratego runtime, and groups options. The stratego section must be enabled when the stratego-runtime section is enabled. The following StrategoRuntimeOption s are supported: Syntax Required? Description Type strategy-package-id = $Expression no Adds a package as a private Stratego package. Can be specified multiple times. Java package identifier interop-registerer-by-reflection = $Expression no Adds an interop registerer to load by reflection. Can be specified multiple times. Java type identifier add-spoofax2-primitives = $Expression no Whether to add the Spoofax 2 Stratego primitives. Boolean add-nabl2-primitives = $Expression no Whether to add the NaBL2 Stratego primitives. Boolean add-statix-primitives = $Expression no Whether to add the Statix Stratego primitives. Boolean with-primitive-library = $Expression no Adds a Stratego primitive strategies library (implementing org.spoofax.interpreter.library.IOperatorRegistry ) to the generated StrategoRuntimeBuilderFactory . The library must have an @Inject constructor. Can be specified multiple times. Java type identifier with-interop-registerer = $Expression no Adds a Stratego interop registerer (implementing org.strategoxt.lang.InteropRegisterer ) to the generated StrategoRuntimeBuilderFactory . The registerer must have an @Inject constructor. Can be specified multiple times. Java type identifier class-kind = $Expression no Specifies whether the classes are generated ( Generated ) or provided manually ( Manual ). Defaults to Generated . Generated or Manual base-StrategoRuntimeBuilderFactory = $Expression no Package and name of the generated StrategoRuntimeBuilderFactory . Java type identifier extend-StrategoRuntimeBuilderFactory = $Expression no Package and name of the extending StrategoRuntimeBuilderFactory , if any. Java type identifier","title":"Stratego runtime"},{"location":"reference/configuration/#completer","text":"The completer { $CompleterOption* } section enables generation of a code completer, and groups options. The constraint-analyzer and statix sections must be enabled when the completer section is enabled. Currently, no CompleterOption s are supported.","title":"Completer"},{"location":"reference/configuration/#reference-resolution","text":"The reference-resolution { $ReferenceResolutionOption* } section enables generation of the reference resolver editor service, and groups options. The following ReferenceResolutionOption s are supported: Syntax Required? Description Type variant = $ReferenceResolutionVariant yes The reference resolution variant to use. n/a The following ReferenceResolutionVariant s are supported: Stratego-based: stratego { strategy = strategy $Strategy } where Strategy is a Stratego strategy, typically editor-resolve .","title":"Reference resolution"},{"location":"reference/configuration/#hover-tooltips","text":"The hover { $HoverOption* } section enables generation of the hover text editor service, and groups options. The following HoverOption s are supported: Syntax Required? Description Type variant = $HoverVariant yes The reference resolution variant to use. n/a The following HoverVariant s are supported: Stratego-based: stratego { strategy = strategy $Strategy } where Strategy is a Stratego strategy, typically editor-hover .","title":"Hover tooltips"},{"location":"reference/configuration/#meta-language-sections","text":"","title":"Meta-language sections"},{"location":"reference/configuration/#sdf3","text":"The sdf3 { $Sdf3Option* } section enables syntax definition with SDF3 . The parser section must be enabled when the sdf3 section is enabled. The following Sdf3Option s are supported: Syntax Required? Description Type source = $Sdf3Source no The source of the SDF3 definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./start.sdf3 as its main file relative to the main source directory. n/a The following $Sdf3Source s are supported: Files: files { $Sdf3FilesOption* } Prebuilt: prebuilt { $Sdf3PrebuiltOption } The following Sdf3FilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main SDF3 file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main SDF3 file relative to the main-source-directory . Defaults to ./start.sdf3 . Path include-directory = $Expression no Adds an include directory from which to resolve SDF3 imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the SDF3 files in it accessable to dependencies. May be given multiple times. Path parse-table-generator { $ParseTableGeneratorOption* } no Parse table generator options. n/a placeholders { $PlaceholderOption* } no Placeholder syntax used in code completion. n/a stratego-concrete-syntax-extension-main-file = $Expression no Sets the main SDF3 file used to create a concrete syntax extension parse table. Path The following $Sdf3PrebuiltOption s are supported: Syntax Required? Description Type parse-table-aterm-file = $Expression yes The prebuilt SDF3 parse table ATerm file to use (usually called sdf.tbl ) relative to the root directory Path parse-table-persisted-file = $Expression yes The prebuilt SDF3 parse table persisted file to use (usually called sdf.bin ) relative to the root directory Path The following ParseTableGeneratorOption s are supported: Syntax Required? Description Type dynamic = $Expression no Whether the generated parse table is dynamic. Defaults to false . Boolean data-dependent = $Expression no Whether the generated parse table is data-dependent. Defaults to false . Boolean layout-sensitive = $Expression no Whether the generated parse table is layout-sensitive. Defaults to false . Boolean solve-deep-conflicts = $Expression no Whether the parse table generator solves deep priority conflicts. Defaults to true . Boolean check-overlap = $Expression no Whether the parse table generator checks for overlap. Defaults to false . Boolean check-priorities = $Expression no Whether the parse table generator checks priorities. Defaults to false . Boolean The following PlaceholderOption s are supported: Syntax Required? Description Type prefix = $Expression no The prefix to a placholder. Defaults to [[ , or empty if the other is defined. String postfix = $Expression no The postfix to a placeholder. Defaults to ]] , or empty if the other is defined. String","title":"SDF3"},{"location":"reference/configuration/#esv","text":"The esv { $EsvOption* } section enables syntax-based styling definition with ESV . The styler section must be enabled when the esv section is enabled. The following EsvOption s are supported: Syntax Required? Description Type source = $EsvSource no The source of the ESV definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.esv as its main file relative to the main source directory. n/a The following $EsvSource s are supported: Files: files { $EsvFilesOption* } Prebuilt: prebuilt { $EsvPrebuiltOption } The following EsvFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main ESV file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main ESV file relative to the main-source-directory . Defaults to ./main.esv . Path include-directory = $Expression no Adds an include directory from which to resolve ESV imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the ESV files in it accessable to dependencies. May be given multiple times. Path The following $EsvPrebuiltOption s are supported: Syntax Required? Description Type file = $Expression yes The prebuilt ESV file to use relative to the root directory Path","title":"ESV"},{"location":"reference/configuration/#statix","text":"The statix { $StatixOption* } section enables static semantics definition with Statix . The constraint-anaylzer section must be enabled when the statix section is enabled. The following StatixOption s are supported: Syntax Required? Description Type source = $StatixSource no The source of the Statix definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.stx as its main file relative to the main source directory. n/a The following $StatixSource s are supported: Files: files { $StatixFilesOption* } Prebuilt: prebuilt { $StatixPrebuiltOption } The following StatixFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main Statix file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main Statix file relative to the main-source-directory . Defaults to ./main.stx . Path include-directory = $Expression no Adds an include directory from which to resolve Statix imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the Statix files in it accessable to dependencies. May be given multiple times. Path sdf3-statix-signature-generation = $Expression no Whether SDF3 to Statix signature generation is enabled. When enabled, stratego { sdf3-statix-explication-generation = true } must also be enabled. Defaults to false . Boolean The following $StatixPrebuiltOption s are supported: Syntax Required? Description Type spec-aterm-directory = $Expression yes The prebuilt Statix spec ATerm directory to use relative to the root directory Path","title":"Statix"},{"location":"reference/configuration/#stratego","text":"The stratego { $StrategoOption* } section enables definition of transformations with Stratego . The stratego-runtime section must be enabled when the stratego section is enabled. The following StrategoOption s are supported: Syntax Required? Description Type source = $StrategoSource no The source of the Statix definition. Defaults to a files source with the top-level source-directory option as its main source directory, and ./main.str2 as its main file relative to the main source directory. n/a output-java-package = $Expression no The Java package into which compiled Stratego Java files are generated. Defaults to the language's package, followed by .strategies . String The following $StrategoSource s are supported: Files: files { $StrategoFilesOption* } The following $StrategoFilesOption s are supported: Syntax Required? Description Type main-source-directory = $Expression no The directory relative to the root directory that contains the main Stratego file. Defaults to the value of the top-level source-directory option. Path main-file = $Expression no The main Stratego file relative to the main-source-directory . Defaults to ./main.str2 . Path include-directory = $Expression no Adds an include directory from which to resolve Stratego imports. May be given multiple times. Path export-directory = $Expression no Exports a directory, making the Stratego files in it accessable to dependencies. May be given multiple times. Path with-import-strategy-package-id = $Expression no Adds a Java package to import in the Java files that have been compiled from Stratego files. For example, \"mb.mylang.strategies\" adds import import mb.mylang.strategies.* . May be given multiple times. String sdf3-statix-explication-generation = $Expression no Whether SDF3 to Statix injection explication/implication generation is enabled. When enabled, statix { sdf3-statix-signature-generation = true } must also be enabled. Defaults to false . Boolean language-strategy-affix = $Expression no The affix that is used to make certain generated strategies unique to the language. This is used both as a prefix and suffix. Defaults to name of the language transformed to a Stratego strategy identifier. Stratego strategy identifier concrete-syntax-extension-parse-table = $Expression no Adds a Stratego concrete syntax extension parse table that can be used to parse Stratego files with. May be given multiple times. Path","title":"Stratego"},{"location":"reference/configuration/#dependencies","text":"The dependencies [ $Dependency* ] section allows specifying dependencies to other language (library) projects. A dependency consists of a dependency expression ( $Exp ) that specifies the source of the dependency. The following expressions are supported: $Coordinate : an exact coordinate to a project, such as org.metaborg:strategolib:1.0.0 . Coordinates are resolved to a statically loaded language, or a dynamically loaded language definition. $CoordinateRequirement : a coordinate to a project with an open version, such as org.metaborg:strategolib:* , resolved identically to $Coordinate except that the latest available version will be chosen. $Path : a relative path (relative to the directory spoofaxc.cfg is in) to a language definition. Dependencies can be of a certain kind ( $DependencyKind ): Build : a dependency that is resolved at build time , meaning when the language definition is built, allowing the use of sources and binaries of the project the dependency points to. Run : a dependency that is resolved at run time , meaning when the language is executed, allowing use of the classes and tasks of the project the dependency points to. Note: this kind of dependency has not yet been implemented. The following Dependency s are supported: Syntax Description $Exp A dependency to a project defined by the expression, available at Build and Run time. $Exp { $DependencyOption* } A dependency to project defined by the expression, with configuration options. The following DependencyOption s are supported: kinds = [ $DependencyKind* ] : set the kinds at which this dependency is resolved. Additionally, build time dependencies can be quickly defined with the build-dependencies [ $BuildDependency* ] section. The following BuildDependency s are supported: Syntax Description $Exp A dependency to project defined by the expression, available only at Build time. For example, the following configuration adds build dependencies to common libraries shipped with Spoofax 3: build-dependencies [ org.metaborg : strategolib:* org.metaborg : gpp:* org.metaborg : libspoofax2:* org.metaborg : libstatix:* ]","title":"Dependencies"},{"location":"reference/configuration/#spoofaxclock","text":"The spoofaxc.lock file, which resides next to the spoofaxc.cfg file, contains values for several options that have defaults derived from other options, in order to keep these derived values stable even when the options they are derived from are changed. For example, when no java-class-id-prefix option is set in spoofaxc.cfg , it will be derived from the name option with some changes to make it compatible as a Java identifier, and is stored under shared.defaultClassPrefix in the spoofaxc.lock file. When you change the name of your language, the stored value will be used, keeping the class prefix the same, making it possible to rename the language without having to rename all class files. Therefore, the spoofaxc.lock file should be checked in to source control, in order to have reproducible builds. If you do want to re-derive a default from other options, remove the option from the spoofaxc.lock file and rebuild the language. The value will be re-derived and stored in spoofaxc.lock , after which you need to check it into source control again.","title":"spoofaxc.lock"},{"location":"reference/language_definition/","text":"Language definition \u00b6 In essence, a language definition in Spoofax 3 consists of several source files along with configuration that specify and implement the various aspects of a language, such as its tokenizer, parser, styler, completer, checker, compiler, commands, and so forth. The source files of a language definition are mostly written in high-level domain-specific meta-languages, with some parts being written in Java. The Spoofax 3 compiler compiles language definitions into language implementations which are essentially standard Java libraries (e.g., JAR files) consisting of Java classes and bundled resources. These classes implement the various aspects of a language, and may use bundled resources such as a parse table which is loaded and interpreted at runtime. In this reference manual we explain the basic anatomy of a language definition, its configuration and file structure, Java classes and how they are instantiated, as well as how a language definition is compiled into a language implementation. Anatomy \u00b6 configuration in CFG syntax in SDF3 styling in ESV static semantics in Statix transformations in Stratego tasks in java (future: PIE DSL) commands, specified in configuration, generated for you menu bindings CLI bindings File structure \u00b6 Java classes \u00b6 description of the Java classes, their instances, and how they are instantiated basic classes: ClassLoaderResources ParseTable/ParserFactory/Parser StylingRules/StylerFactory/Styler StrategoRuntimeBuilderFactory ConstraintAnalyzerFactory/ConstraintAnalyzer components: ResourcesComponent/ResourcesModule Component/Module Scope/Qualifier how they are instantiated by Dagger LanguageInstance: tasks command defs auto command request CLI commands menu items task definitions: Tokenize Style Completion Check/CheckMulti/CheckAggregator/CheckDeaggregator Parse Analyze/AnalyzeMulti GetStrategoRuntimeProvider commands: Instantiation \u00b6 dependency injection Compilation \u00b6 description of how a language definition is compiled into a language implementation, and what the compiled form looks like. generate Java sources into: build/generated/sources/language build/generated/sources/adapter generate Stratego sources into: build/generated/sources/languageSpecification/stratego","title":"Language definition"},{"location":"reference/language_definition/#language-definition","text":"In essence, a language definition in Spoofax 3 consists of several source files along with configuration that specify and implement the various aspects of a language, such as its tokenizer, parser, styler, completer, checker, compiler, commands, and so forth. The source files of a language definition are mostly written in high-level domain-specific meta-languages, with some parts being written in Java. The Spoofax 3 compiler compiles language definitions into language implementations which are essentially standard Java libraries (e.g., JAR files) consisting of Java classes and bundled resources. These classes implement the various aspects of a language, and may use bundled resources such as a parse table which is loaded and interpreted at runtime. In this reference manual we explain the basic anatomy of a language definition, its configuration and file structure, Java classes and how they are instantiated, as well as how a language definition is compiled into a language implementation.","title":"Language definition"},{"location":"reference/language_definition/#anatomy","text":"configuration in CFG syntax in SDF3 styling in ESV static semantics in Statix transformations in Stratego tasks in java (future: PIE DSL) commands, specified in configuration, generated for you menu bindings CLI bindings","title":"Anatomy"},{"location":"reference/language_definition/#file-structure","text":"","title":"File structure"},{"location":"reference/language_definition/#java-classes","text":"description of the Java classes, their instances, and how they are instantiated basic classes: ClassLoaderResources ParseTable/ParserFactory/Parser StylingRules/StylerFactory/Styler StrategoRuntimeBuilderFactory ConstraintAnalyzerFactory/ConstraintAnalyzer components: ResourcesComponent/ResourcesModule Component/Module Scope/Qualifier how they are instantiated by Dagger LanguageInstance: tasks command defs auto command request CLI commands menu items task definitions: Tokenize Style Completion Check/CheckMulti/CheckAggregator/CheckDeaggregator Parse Analyze/AnalyzeMulti GetStrategoRuntimeProvider commands:","title":"Java classes"},{"location":"reference/language_definition/#instantiation","text":"dependency injection","title":"Instantiation"},{"location":"reference/language_definition/#compilation","text":"description of how a language definition is compiled into a language implementation, and what the compiled form looks like. generate Java sources into: build/generated/sources/language build/generated/sources/adapter generate Stratego sources into: build/generated/sources/languageSpecification/stratego","title":"Compilation"},{"location":"reference/eclipse-lwb/eclipse-project-files/","text":"Eclipse Project Files \u00b6 To be able to import a Spoofax 3 project into Eclipse, it should have at least the .project and .classpath project files. Those files live in the root of the project (i.e., where the spoofaxc.cfg file lives), with the following minimum content: Eclipse will adjust these files as needed. Ensure these files are not in .gitignore Often these Eclipse files are ignored for version control by specifying them in .gitignore . To allow these projects to be imported in the future, do not ignore them when committing the files. Adjust the myproject to be the name of the project. .project myproject spoofax.lwb.eclipse.builder.project.references spoofax.lwb.eclipse.builder clean,full,incremental, org.eclipse.jdt.core.javabuilder org.eclipse.jdt.core.javanature spoofax.lwb.eclipse.nature .classpath ","title":"Eclipse Project Files"},{"location":"reference/eclipse-lwb/eclipse-project-files/#eclipse-project-files","text":"To be able to import a Spoofax 3 project into Eclipse, it should have at least the .project and .classpath project files. Those files live in the root of the project (i.e., where the spoofaxc.cfg file lives), with the following minimum content: Eclipse will adjust these files as needed. Ensure these files are not in .gitignore Often these Eclipse files are ignored for version control by specifying them in .gitignore . To allow these projects to be imported in the future, do not ignore them when committing the files. Adjust the myproject to be the name of the project. .project myproject spoofax.lwb.eclipse.builder.project.references spoofax.lwb.eclipse.builder clean,full,incremental, org.eclipse.jdt.core.javabuilder org.eclipse.jdt.core.javanature spoofax.lwb.eclipse.nature .classpath ","title":"Eclipse Project Files"},{"location":"release/download/","text":"Downloads \u00b6 This page contains download links to the latest release and development version of Spoofax 3. Latest release \u00b6 The latest release of Spoofax is 0.22.0, released on 27-05-2024. We recommend the use of releases because we make sure to make them available indefinitely. Follow the Installation tutorial for instructions on how to install and run Spoofax. Eclipse Language Workbench Environment \u00b6 With embedded JVM: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM Without embedded JVM: Windows 64-bit macOS 64-bit Linux 64-bit Repository for installing into an existing Eclipse installation: https://artifacts.metaborg.org/content/unzip/releases-unzipped/org/metaborg/spoofax.lwb.eclipse.repository/0.22.0/spoofax.lwb.eclipse.repository-0.22.0.zip-unzip/ Development version \u00b6 The development version of Spoofax is always develop-SNAPSHOT . Development versions of Spoofax use snapshot versioning, meaning that they always point to the latest build that is made whenever a change is pushed to Spoofax\u2019s repositories. Therefore, older builds are not available since they are continuously replaced by newer builds. We do not recommend usage of the development version unless you absolutely need its features, want to help us test, or if you know what you are doing. Eclipse Language Workbench Environment \u00b6 With embedded JVM: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM Without embedded JVM: Windows 64-bit macOS 64-bit Linux 64-bit","title":"Downloads"},{"location":"release/download/#downloads","text":"This page contains download links to the latest release and development version of Spoofax 3.","title":"Downloads"},{"location":"release/download/#latest-release","text":"The latest release of Spoofax is 0.22.0, released on 27-05-2024. We recommend the use of releases because we make sure to make them available indefinitely. Follow the Installation tutorial for instructions on how to install and run Spoofax.","title":"Latest release"},{"location":"release/download/#eclipse-language-workbench-environment","text":"With embedded JVM: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM Without embedded JVM: Windows 64-bit macOS 64-bit Linux 64-bit Repository for installing into an existing Eclipse installation: https://artifacts.metaborg.org/content/unzip/releases-unzipped/org/metaborg/spoofax.lwb.eclipse.repository/0.22.0/spoofax.lwb.eclipse.repository-0.22.0.zip-unzip/","title":"Eclipse Language Workbench Environment"},{"location":"release/download/#development-version","text":"The development version of Spoofax is always develop-SNAPSHOT . Development versions of Spoofax use snapshot versioning, meaning that they always point to the latest build that is made whenever a change is pushed to Spoofax\u2019s repositories. Therefore, older builds are not available since they are continuously replaced by newer builds. We do not recommend usage of the development version unless you absolutely need its features, want to help us test, or if you know what you are doing.","title":"Development version"},{"location":"release/download/#eclipse-language-workbench-environment_1","text":"With embedded JVM: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM Without embedded JVM: Windows 64-bit macOS 64-bit Linux 64-bit","title":"Eclipse Language Workbench Environment"},{"location":"tutorial/add_transformation/","text":"Adding a transformation \u00b6 Finally, we will define a transformation for our language and add a task, command-tasks, command, and menu item for it. Open the main Stratego file helloworld/src/main.str . Stratego . is a meta-language for defining term (AST) transformations through rewrite rules. We will add a silly transformation that replaces all instances of World () with Hello () . Add the following code to the end of the Stratego file: rules replace-world : Hello () - > Hello () replace-world : World () - > Hello () replace-worlds = topdown ( try ( replace-world )) The replace-world rule passes Hello () terms but rewrites World () terms to Hello () . The replace-worlds strategy tries to apply replace-world in a top-down manner over the entire AST. src/main.str full contents module main imports statixruntime statix / api injections / - signatures / - rules // Analysis pre-analyze = explicate-injections-helloworld-Start post-analyze = implicate-injections-helloworld-Start editor-analyze = stx-editor-analyze ( pre-analyze , post-analyze | \"main\" , \"programOk\" ) rules replace-world : Hello () - > Hello () replace-world : World () - > Hello () replace-worlds = topdown ( try ( replace-world )) ``` Now we add a task and command-task for this transformation . We define two separate tasks to keep separate 1 . the act of transforming the program , and 2 . feeding back the result of that transformation to the user that executes a command . This practice later allows us to reuse the first task in a different task if we need to . Right-click the ` mb . helloworld . task ` package and create the ` HelloWorldReplaceWorlds ` class and replace the entire Java file with : ``` { . java . annotate linenums = \"1\" } package mb . helloworld . task ; import mb . helloworld . HelloWorldClassLoaderResources ; import mb . helloworld . HelloWorldScope ; import mb . pie . api . ExecContext ; import mb . pie . api . stamp . resource . ResourceStampers ; import mb . stratego . pie . AstStrategoTransformTaskDef ; import javax . inject . Inject ; import java . io . IOException ; @ HelloWorldScope public class HelloWorldReplaceWorlds extends AstStrategoTransformTaskDef { private final HelloWorldClassLoaderResources classloaderResources ; @ Inject public HelloWorldReplaceWorlds ( // 1 HelloWorldClassLoaderResources classloaderResources , HelloWorldGetStrategoRuntimeProvider getStrategoRuntimeProvider ) { super ( getStrategoRuntimeProvider , \"replace-worlds\" ); // 2 this . classloaderResources = classloaderResources ; } @ Override public String getId () { // 3 return getClass () . getName (); } @ Override protected void createDependencies ( ExecContext context ) throws IOException { // 4 context . require ( classloaderResources . tryGetAsLocalResource ( getClass ()), ResourceStampers . hashFile ()); } } This task extends AstStrategoTransformTaskDef which is a convenient abstract class for creating tasks that run Stratego transformations by implementing a constructor and a couple of methods: The constructor should inject HelloWorldClassLoaderResources which we again will use to create a self-dependency, and HelloWorldGetStrategoRuntimeProvider which is a task that Spoofax generates for your language, which provides a Stratego runtime to execute strategies with. The HelloWorldGetStrategoRuntimeProvider instance is provided to the superclass constructor, along with the strategy that we want this task to execute, which is \"replace-worlds\" . We override getId of TaskDef again to give this task a unique identifier. We override createDependencies of AstStrategoTransformTaskDef to create a self-dependency. Then create the HelloWorldShowReplaceWorlds class and replace the entire Java file with: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 package mb.helloworld.task ; import java.io.Serializable ; import java.util.Objects ; import javax.inject.Inject ; import org.checkerframework.checker.nullness.qual.Nullable ; import mb.helloworld.HelloWorldClassLoaderResources ; import mb.helloworld.HelloWorldScope ; import mb.pie.api.ExecContext ; import mb.pie.api.TaskDef ; import mb.pie.api.stamp.resource.ResourceStampers ; import mb.resource.ResourceKey ; import mb.spoofax.core.language.command.CommandFeedback ; import mb.spoofax.core.language.command.ShowFeedback ; import mb.aterm.common.TermToString ; @HelloWorldScope public class HelloWorldShowReplaceWorlds implements TaskDef < HelloWorldShowReplaceWorlds . Args , CommandFeedback > { public static class Args implements Serializable { private static final long serialVersionUID = 1L ; public final ResourceKey file ; public Args ( ResourceKey file ) { this . file = file ; } @Override public boolean equals ( @Nullable Object o ) { if ( this == o ) return true ; if ( o == null || getClass () != o . getClass ()) return false ; final Args args = ( Args ) o ; return file . equals ( args . file ); } @Override public int hashCode () { return Objects . hash ( file ); } @Override public String toString () { return \"Args{\" + \"file=\" + file + '}' ; } } private final HelloWorldClassLoaderResources classloaderResources ; private final HelloWorldParse parse ; private final HelloWorldReplaceWorlds replaceWorlds ; @Inject public HelloWorldShowReplaceWorlds ( HelloWorldClassLoaderResources classloaderResources , HelloWorldParse parse , HelloWorldReplaceWorlds replaceWorlds ) { this . classloaderResources = classloaderResources ; this . parse = parse ; this . replaceWorlds = replaceWorlds ; } @Override public CommandFeedback exec ( ExecContext context , Args args ) throws Exception { context . require ( classloaderResources . tryGetAsLocalResource ( getClass ()), ResourceStampers . hashFile ()); final ResourceKey file = args . file ; return context . require ( replaceWorlds , parse . inputBuilder (). withFile ( file ). buildAstSupplier ()). mapOrElse ( ast -> CommandFeedback . of ( ShowFeedback . showText ( TermToString . toString ( ast ), \"Replaced World()s with Hello()s for '\" + file + \"'\" )), e -> CommandFeedback . ofTryExtractMessagesFrom ( e , file ) ); } @Override public String getId () { return getClass (). getName (); } } This class very similar to HelloWorldShowParsedAst , but runs the HelloWorldReplaceWorlds task on the parsed AST, transforming the AST. Now open helloworld/spoofax.cfg again and register the tasks by adding: task-def mb.helloworld.task.HelloWorldReplaceWorlds let showReplaceWorlds = task-def mb.helloworld.task.HelloWorldShowReplaceWorlds Then add a command for it by adding: let showReplaceWorldsCommand = command-def { task-def = showReplaceWorlds display-name = \"Replace world with hello\" parameters = [ file = parameter { type = java mb.resource.ResourceKey argument-providers = [Context(ReadableResource)] } ] } Finally, add menu items for the command by adding: editor-context-menu [ menu \"Transform\" [ command-action { command-def = showReplaceWorldsCommand execution-type = Once } command-action { command-def = showReplaceWorldsCommand execution-type = Continuous } ] ] resource-context-menu [ menu \"Transform\" [ command-action { command-def = showReplaceWorldsCommand execution-type = Once required-resource-types = [File] } ] ] Build the project so that we can test our changes. Test the command similarly to testing the \"Show parsed AST\" command.","title":"Adding a transformation"},{"location":"tutorial/add_transformation/#adding-a-transformation","text":"Finally, we will define a transformation for our language and add a task, command-tasks, command, and menu item for it. Open the main Stratego file helloworld/src/main.str . Stratego . is a meta-language for defining term (AST) transformations through rewrite rules. We will add a silly transformation that replaces all instances of World () with Hello () . Add the following code to the end of the Stratego file: rules replace-world : Hello () - > Hello () replace-world : World () - > Hello () replace-worlds = topdown ( try ( replace-world )) The replace-world rule passes Hello () terms but rewrites World () terms to Hello () . The replace-worlds strategy tries to apply replace-world in a top-down manner over the entire AST. src/main.str full contents module main imports statixruntime statix / api injections / - signatures / - rules // Analysis pre-analyze = explicate-injections-helloworld-Start post-analyze = implicate-injections-helloworld-Start editor-analyze = stx-editor-analyze ( pre-analyze , post-analyze | \"main\" , \"programOk\" ) rules replace-world : Hello () - > Hello () replace-world : World () - > Hello () replace-worlds = topdown ( try ( replace-world )) ``` Now we add a task and command-task for this transformation . We define two separate tasks to keep separate 1 . the act of transforming the program , and 2 . feeding back the result of that transformation to the user that executes a command . This practice later allows us to reuse the first task in a different task if we need to . Right-click the ` mb . helloworld . task ` package and create the ` HelloWorldReplaceWorlds ` class and replace the entire Java file with : ``` { . java . annotate linenums = \"1\" } package mb . helloworld . task ; import mb . helloworld . HelloWorldClassLoaderResources ; import mb . helloworld . HelloWorldScope ; import mb . pie . api . ExecContext ; import mb . pie . api . stamp . resource . ResourceStampers ; import mb . stratego . pie . AstStrategoTransformTaskDef ; import javax . inject . Inject ; import java . io . IOException ; @ HelloWorldScope public class HelloWorldReplaceWorlds extends AstStrategoTransformTaskDef { private final HelloWorldClassLoaderResources classloaderResources ; @ Inject public HelloWorldReplaceWorlds ( // 1 HelloWorldClassLoaderResources classloaderResources , HelloWorldGetStrategoRuntimeProvider getStrategoRuntimeProvider ) { super ( getStrategoRuntimeProvider , \"replace-worlds\" ); // 2 this . classloaderResources = classloaderResources ; } @ Override public String getId () { // 3 return getClass () . getName (); } @ Override protected void createDependencies ( ExecContext context ) throws IOException { // 4 context . require ( classloaderResources . tryGetAsLocalResource ( getClass ()), ResourceStampers . hashFile ()); } } This task extends AstStrategoTransformTaskDef which is a convenient abstract class for creating tasks that run Stratego transformations by implementing a constructor and a couple of methods: The constructor should inject HelloWorldClassLoaderResources which we again will use to create a self-dependency, and HelloWorldGetStrategoRuntimeProvider which is a task that Spoofax generates for your language, which provides a Stratego runtime to execute strategies with. The HelloWorldGetStrategoRuntimeProvider instance is provided to the superclass constructor, along with the strategy that we want this task to execute, which is \"replace-worlds\" . We override getId of TaskDef again to give this task a unique identifier. We override createDependencies of AstStrategoTransformTaskDef to create a self-dependency. Then create the HelloWorldShowReplaceWorlds class and replace the entire Java file with: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 package mb.helloworld.task ; import java.io.Serializable ; import java.util.Objects ; import javax.inject.Inject ; import org.checkerframework.checker.nullness.qual.Nullable ; import mb.helloworld.HelloWorldClassLoaderResources ; import mb.helloworld.HelloWorldScope ; import mb.pie.api.ExecContext ; import mb.pie.api.TaskDef ; import mb.pie.api.stamp.resource.ResourceStampers ; import mb.resource.ResourceKey ; import mb.spoofax.core.language.command.CommandFeedback ; import mb.spoofax.core.language.command.ShowFeedback ; import mb.aterm.common.TermToString ; @HelloWorldScope public class HelloWorldShowReplaceWorlds implements TaskDef < HelloWorldShowReplaceWorlds . Args , CommandFeedback > { public static class Args implements Serializable { private static final long serialVersionUID = 1L ; public final ResourceKey file ; public Args ( ResourceKey file ) { this . file = file ; } @Override public boolean equals ( @Nullable Object o ) { if ( this == o ) return true ; if ( o == null || getClass () != o . getClass ()) return false ; final Args args = ( Args ) o ; return file . equals ( args . file ); } @Override public int hashCode () { return Objects . hash ( file ); } @Override public String toString () { return \"Args{\" + \"file=\" + file + '}' ; } } private final HelloWorldClassLoaderResources classloaderResources ; private final HelloWorldParse parse ; private final HelloWorldReplaceWorlds replaceWorlds ; @Inject public HelloWorldShowReplaceWorlds ( HelloWorldClassLoaderResources classloaderResources , HelloWorldParse parse , HelloWorldReplaceWorlds replaceWorlds ) { this . classloaderResources = classloaderResources ; this . parse = parse ; this . replaceWorlds = replaceWorlds ; } @Override public CommandFeedback exec ( ExecContext context , Args args ) throws Exception { context . require ( classloaderResources . tryGetAsLocalResource ( getClass ()), ResourceStampers . hashFile ()); final ResourceKey file = args . file ; return context . require ( replaceWorlds , parse . inputBuilder (). withFile ( file ). buildAstSupplier ()). mapOrElse ( ast -> CommandFeedback . of ( ShowFeedback . showText ( TermToString . toString ( ast ), \"Replaced World()s with Hello()s for '\" + file + \"'\" )), e -> CommandFeedback . ofTryExtractMessagesFrom ( e , file ) ); } @Override public String getId () { return getClass (). getName (); } } This class very similar to HelloWorldShowParsedAst , but runs the HelloWorldReplaceWorlds task on the parsed AST, transforming the AST. Now open helloworld/spoofax.cfg again and register the tasks by adding: task-def mb.helloworld.task.HelloWorldReplaceWorlds let showReplaceWorlds = task-def mb.helloworld.task.HelloWorldShowReplaceWorlds Then add a command for it by adding: let showReplaceWorldsCommand = command-def { task-def = showReplaceWorlds display-name = \"Replace world with hello\" parameters = [ file = parameter { type = java mb.resource.ResourceKey argument-providers = [Context(ReadableResource)] } ] } Finally, add menu items for the command by adding: editor-context-menu [ menu \"Transform\" [ command-action { command-def = showReplaceWorldsCommand execution-type = Once } command-action { command-def = showReplaceWorldsCommand execution-type = Continuous } ] ] resource-context-menu [ menu \"Transform\" [ command-action { command-def = showReplaceWorldsCommand execution-type = Once required-resource-types = [File] } ] ] Build the project so that we can test our changes. Test the command similarly to testing the \"Show parsed AST\" command.","title":"Adding a transformation"},{"location":"tutorial/change_static_semantics/","text":"Changing the static semantics \u00b6 Now we will change the static semantics of the language. Open the main Statix file helloworld/src/main.stx Statix is a meta-language for defining the static semantics of your language, which includes type checking. First we will update the Statix specification to handle the new language constructs. Replace the programOk ( _ ). line with programOk(Program(parts)) :- partsOk(parts). , meaning that we accept programs consisting of parts, as long as their parts are ok. As a silly rule, we will add a warning to all instances of world in the program. Add the following code to the end of the Statix definition: partOk : Part partOk ( Hello ()). partOk ( World ()) : - try { false } | warning $[ World !] . partsOk maps partOk ( list ( * )) This adds a partOk rule that lets all Hello () parts pass, but will add a warning to all World () parts. partsOk goes over a list of parts and applies partOk . src/main.stx full contents module main imports signatures / start-sig rules programOk : Start programOk ( Program ( parts )) : - partsOk ( parts ). partOk : Part partOk ( Hello ()). partOk ( World ()) : - try { false } | warning $[ World !] . partsOk maps partOk ( list ( * )) Build the project, and a warning marker should appear under all instances of world in the example program.","title":"Change static semantics"},{"location":"tutorial/change_static_semantics/#changing-the-static-semantics","text":"Now we will change the static semantics of the language. Open the main Statix file helloworld/src/main.stx Statix is a meta-language for defining the static semantics of your language, which includes type checking. First we will update the Statix specification to handle the new language constructs. Replace the programOk ( _ ). line with programOk(Program(parts)) :- partsOk(parts). , meaning that we accept programs consisting of parts, as long as their parts are ok. As a silly rule, we will add a warning to all instances of world in the program. Add the following code to the end of the Statix definition: partOk : Part partOk ( Hello ()). partOk ( World ()) : - try { false } | warning $[ World !] . partsOk maps partOk ( list ( * )) This adds a partOk rule that lets all Hello () parts pass, but will add a warning to all World () parts. partsOk goes over a list of parts and applies partOk . src/main.stx full contents module main imports signatures / start-sig rules programOk : Start programOk ( Program ( parts )) : - partsOk ( parts ). partOk : Part partOk ( Hello ()). partOk ( World ()) : - try { false } | warning $[ World !] . partsOk maps partOk ( list ( * )) Build the project, and a warning marker should appear under all instances of world in the example program.","title":"Changing the static semantics"},{"location":"tutorial/create_language_project/","text":"Creating a language project \u00b6 This tutorial gets you started with language development by creating a language project and changing various aspects of the language. First follow the installation tutorial if you haven't done so yet. Creating a new project \u00b6 In Eclipse, open the new project dialog by choosing File \u2023 New \u2023 Project from the main menu. In the new project dialog, select Spoofax LWB \u2023 Spoofax language project and press Next . In this wizard, you can customize the various names your language will use. However, for the purpose of this tutorial, fill in HelloWorld as the name of the project, which will automatically fill in the other elements with defaults. Then press Finish to create the project. There should now be a project named helloworld in the Package Explorer . Adding syntax \u00b6 First we will add some syntax to the language. Open the main SDF3 file helloworld/src/start.sdf3 file by expanding the folders and double-clicking the file. SDF3 is a meta-language (i.e., a language to describe languages) for describing the syntax of a language, from which Spoofax will derive the parser of your language. Under the context-free syntax section, replace the Start . Empty = <> line with Start . Program = <<{ Part \" \" } * >> , indicating that the language accepts programs which consists of zero or more parts. Part is a sort and must be defined by adding its name to the context-free sorts section on a new line. Now we will add syntax productions to Part to the context-free syntax section. Add Part . Hello = < hello > on a new line, indicating that one sort of Part is the word hello. Then add Part . World = < world > on a new line, indicating that one sort of Part is the word world . src/start.sdf3 full contents module start context-free start-symbols Start context-free sorts Start Part context-free syntax Start . Program = <<{ Part \" \" } * >> Part . Hello = < hello > Part . World = < world > lexical syntax LAYOUT = [ \\ \\ n \\ v \\ f \\ r ] context-free restrictions LAYOUT ? - / - [ \\ \\ n \\ v \\ f \\ r ] To observe our changes, build the project by clicking on the project in the Package Explorer and choosing Project \u2023 Build Project from the main menu, or by pressing Cmd + B on macOS or Ctrl + B on others. To see when the build is done, open the progress window by choosing Window \u2023 Show View \u2023 Progress . If the progress view is empty, the build is done. The initial build can be a bit slow because there is a lot of code to compile in the background. Subsequent builds will be faster due to incrementalization. Create an example file for your language by right-clicking the project and choosing New \u2023 File , filling in example/example1.hel as file name, and pressing Finish . Type a valid sentence such as hello world hello hello world in this file, and it will highlight purple indicating that hello and world are keywords. We can also run a debugging command on the example file to check the AST that the parser produces. There are three ways to run this debugging command: Make sure the example1.hel editor is open and active (i.e., has focus), and choose from the main menu: Spoofax \u2023 Debug \u2023 Show parsed AST . Make sure the example1.hel editor is open and active (i.e., has focus), right-click in the editor and choose: HelloWorld \u2023 Debug \u2023 Show parsed AST . Right-click the example/example1.hel file in the Package Explorer and choose: HelloWorld \u2023 Debug \u2023 Show parsed AST Running this command will open a new editor with the AST of the program, such as: Program ([ Hello (), World ()]) There are also (continuous) variants of these debugging commands which will keep updating the debugging view when the source program changes. You can drag tabs to the sides of the screen to keep multiple editors open simultaneously, for example to keep the continuous debugging view visible, and to keep the syntax definition files editable. Warning There is currently a bug where continuous debugging views are not updated any more after the language is rebuilt. In that case, you have to open the continuous debugging view again. Testing the new syntax \u00b6 We can also systematically test the syntax (and other facets) of the language with the Spoofax Testing Language (SPT) . Open the SPT file helloworld/test/test.spt . This file contains one test which tests that the empty program parses successfully, which is still the case because a program can consist of 0 parts. Add a new test case to the test suite by adding: test hello world parse [[ hello world ]] parse succeeds which tests that hello world parses successfully. You can also add negative tests such as: test gibberish [[ asdfasdfasdf ]] parse fails test/test.spt full contents module test test parse empty [[]] parse succeeds test hello world parse [[ hello world ]] parse succeeds test gibberish [[ asdfasdfasdf ]] parse fails If you keep the SPT file open and rebuild your language, the SPT tests will be re-executed to provide feedback whether your change to the language conforms to your tests. You can also run all SPT tests by right-clicking a directory with SPT files, or by right-clicking the language project, and choosing SPT \u2023 Run SPT tests . This will (once the tests are done executing) pop up a SPT Test Runner view with the results of testing. Changing syntax highlighting \u00b6 Now we will change the syntax highlighter of the language. Open the main ESV file helloworld/src/main.esv . ESV is a meta-language for describing the syntax highlighter. Change the keyword : 127 0 85 bold line to keyword : 0 0 150 bold and build the project. Then check your example1.hel example file, it should now be highlighted blue. To make iteration easier, you can drag the example1.hel tab to the side of the screen to open the language definition and example file side-by-side. You can play around with the coloring a bit and choose a style to your liking. Remember to rebuild the project after making a change to the language definition. Troubleshooting \u00b6 Check the troubleshooting guide if you run into problems.","title":"Creating a language project"},{"location":"tutorial/create_language_project/#creating-a-language-project","text":"This tutorial gets you started with language development by creating a language project and changing various aspects of the language. First follow the installation tutorial if you haven't done so yet.","title":"Creating a language project"},{"location":"tutorial/create_language_project/#creating-a-new-project","text":"In Eclipse, open the new project dialog by choosing File \u2023 New \u2023 Project from the main menu. In the new project dialog, select Spoofax LWB \u2023 Spoofax language project and press Next . In this wizard, you can customize the various names your language will use. However, for the purpose of this tutorial, fill in HelloWorld as the name of the project, which will automatically fill in the other elements with defaults. Then press Finish to create the project. There should now be a project named helloworld in the Package Explorer .","title":"Creating a new project"},{"location":"tutorial/create_language_project/#adding-syntax","text":"First we will add some syntax to the language. Open the main SDF3 file helloworld/src/start.sdf3 file by expanding the folders and double-clicking the file. SDF3 is a meta-language (i.e., a language to describe languages) for describing the syntax of a language, from which Spoofax will derive the parser of your language. Under the context-free syntax section, replace the Start . Empty = <> line with Start . Program = <<{ Part \" \" } * >> , indicating that the language accepts programs which consists of zero or more parts. Part is a sort and must be defined by adding its name to the context-free sorts section on a new line. Now we will add syntax productions to Part to the context-free syntax section. Add Part . Hello = < hello > on a new line, indicating that one sort of Part is the word hello. Then add Part . World = < world > on a new line, indicating that one sort of Part is the word world . src/start.sdf3 full contents module start context-free start-symbols Start context-free sorts Start Part context-free syntax Start . Program = <<{ Part \" \" } * >> Part . Hello = < hello > Part . World = < world > lexical syntax LAYOUT = [ \\ \\ n \\ v \\ f \\ r ] context-free restrictions LAYOUT ? - / - [ \\ \\ n \\ v \\ f \\ r ] To observe our changes, build the project by clicking on the project in the Package Explorer and choosing Project \u2023 Build Project from the main menu, or by pressing Cmd + B on macOS or Ctrl + B on others. To see when the build is done, open the progress window by choosing Window \u2023 Show View \u2023 Progress . If the progress view is empty, the build is done. The initial build can be a bit slow because there is a lot of code to compile in the background. Subsequent builds will be faster due to incrementalization. Create an example file for your language by right-clicking the project and choosing New \u2023 File , filling in example/example1.hel as file name, and pressing Finish . Type a valid sentence such as hello world hello hello world in this file, and it will highlight purple indicating that hello and world are keywords. We can also run a debugging command on the example file to check the AST that the parser produces. There are three ways to run this debugging command: Make sure the example1.hel editor is open and active (i.e., has focus), and choose from the main menu: Spoofax \u2023 Debug \u2023 Show parsed AST . Make sure the example1.hel editor is open and active (i.e., has focus), right-click in the editor and choose: HelloWorld \u2023 Debug \u2023 Show parsed AST . Right-click the example/example1.hel file in the Package Explorer and choose: HelloWorld \u2023 Debug \u2023 Show parsed AST Running this command will open a new editor with the AST of the program, such as: Program ([ Hello (), World ()]) There are also (continuous) variants of these debugging commands which will keep updating the debugging view when the source program changes. You can drag tabs to the sides of the screen to keep multiple editors open simultaneously, for example to keep the continuous debugging view visible, and to keep the syntax definition files editable. Warning There is currently a bug where continuous debugging views are not updated any more after the language is rebuilt. In that case, you have to open the continuous debugging view again.","title":"Adding syntax"},{"location":"tutorial/create_language_project/#testing-the-new-syntax","text":"We can also systematically test the syntax (and other facets) of the language with the Spoofax Testing Language (SPT) . Open the SPT file helloworld/test/test.spt . This file contains one test which tests that the empty program parses successfully, which is still the case because a program can consist of 0 parts. Add a new test case to the test suite by adding: test hello world parse [[ hello world ]] parse succeeds which tests that hello world parses successfully. You can also add negative tests such as: test gibberish [[ asdfasdfasdf ]] parse fails test/test.spt full contents module test test parse empty [[]] parse succeeds test hello world parse [[ hello world ]] parse succeeds test gibberish [[ asdfasdfasdf ]] parse fails If you keep the SPT file open and rebuild your language, the SPT tests will be re-executed to provide feedback whether your change to the language conforms to your tests. You can also run all SPT tests by right-clicking a directory with SPT files, or by right-clicking the language project, and choosing SPT \u2023 Run SPT tests . This will (once the tests are done executing) pop up a SPT Test Runner view with the results of testing.","title":"Testing the new syntax"},{"location":"tutorial/create_language_project/#changing-syntax-highlighting","text":"Now we will change the syntax highlighter of the language. Open the main ESV file helloworld/src/main.esv . ESV is a meta-language for describing the syntax highlighter. Change the keyword : 127 0 85 bold line to keyword : 0 0 150 bold and build the project. Then check your example1.hel example file, it should now be highlighted blue. To make iteration easier, you can drag the example1.hel tab to the side of the screen to open the language definition and example file side-by-side. You can play around with the coloring a bit and choose a style to your liking. Remember to rebuild the project after making a change to the language definition.","title":"Changing syntax highlighting"},{"location":"tutorial/create_language_project/#troubleshooting","text":"Check the troubleshooting guide if you run into problems.","title":"Troubleshooting"},{"location":"tutorial/install/","text":"Installing the Spoofax 3 language workbench \u00b6 This tutorial gets you set up for language development in Spoofax 3 by installing the Spoofax 3 Eclipse LWB environment. Requirements \u00b6 Spoofax 3 runs on the major operating systems: Windows (64 bits) macOS (64 bits) Linux (64 bits) Other than that, nothing is required as everything is contained in the archive we are going to download. Download \u00b6 To get started, we will download a premade Eclipse installation that comes bundled with the Spoofax 3 LWB plugin. We will download version 0.22.0 released on 27-05-2024. Download the latest version for your platform: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM These are bundled with an embedded JVM so that you do not need to have a JVM installed. If your system has a JVM of version 17 (exactly, higher versions are not supported), and would rather use that, use the following download links instead: Windows 64-bit macOS 64-bit Linux 64-bit Unpack \u00b6 Unpack the downloaded archive to a directory with write access . Write access is required because Eclipse needs to write to several configuration files inside its installation. Warning The unpacked directory or application may be renamed, but do not include spaces or other characters that would not be allowed in a URI (i.e., : ? # [ ] @ ). The same is true for the directory the archive is extracted to. Failing to do so breaks a built-in classpath detection mechanism which will cause Java compilation errors. Warning On Windows do not unpack the Eclipse installation into Program Files , because no write access is granted there, breaking both Eclipse and Spoofax. On macOS Sierra (10.12) and above, after unpacking, open the Terminal and navigate to the directory where the Spoofax3.app file is located and execute: xattr -rc Spoofax3.app Running Eclipse \u00b6 Start up Eclipse, depending on your operating system: Windows: run Spoofax3/eclipse.exe macOS run Spoofax3.app Linux run Spoofax3/eclipse Warning macOS Sierra (10.12) and above will mark the unpacked Spoofax3.app as \"damaged\" due to a modified signed/notarized application, because we have modified the eclipse.ini file inside it. To fix this, open the Terminal, navigate to the directory where the Spoofax3.app file is located, and execute: xattr -rc Spoofax3.app After starting up, choose where your workspace will be stored. The Eclipse workspace will contain all of your settings, and is the default location for new projects. Warning Currently, there are several bugs regarding spaces in the workspace path, so ensure there are no spaces on the workspace path. Configuring Eclipse's preferences \u00b6 Some Eclipse preferences unfortunately have sub-optimal defaults. After you have chosen a workspace and Eclipse has completely started up (and you have closed the Welcome page), go to the Eclipse preferences by pressing Cmd + , on macOS and by going to the Window \u2023 Preferences menu on others, and set these options: General \u2023 Startup and Shutdown Enable: Refresh workspace on startup General \u2023 Workspace Enable: Refresh using native hooks or polling General \u2023 Workspace \u2023 Build Enable: Save automatically before manual build We need to make sure that Eclipse has detected an installed JRE . Open the Eclipse preferences and go to the Java \u2023 Installed JREs page: If there are no installed JREs, and you've downloaded an Eclipse installation with an embedded JVM , press Search... and navigate to the location where you unpacked the Eclipse installation, and choose the jvm directory in it. Then press the checkmark of the JRE to activate it. If there are no installed JREs, and you've downloaded an Eclipse installation without an embedded JVM , press Search... and navigate to the location where your JVM installed, and choose it. Then press the checkmark of the JRE to activate it. If there are one or more installed JVMs, but none are selected, select an appropriate one by pressing the checkmark. If there are one or more installed JVMs, and one is selected, you are good to go. Finally, you may configure Eclipse to your liking. Some typical settings to adjust: General \u2023 Editors \u2023 Text Editors Displayed tab width : change to your desired tab width. Most of Spoofax uses 2 by convention. Insert spaces for tabs : enable, as Spoofax uses spaces by convention. Show print margin : enable and set if you want to set a maximum line length. Show line numbers : enable if you want to see line numbers. General \u2023 Appearance : choose a Theme to your liking. General \u2023 Appearance \u2023 Colors and Fons \u2023 Text Font : the JetBrains Mono font supports ligatures and is used in (most of) the lecture slides (install separately). General \u2023 Keys : change keybindings. Tip These preferences are stored per workspace. If you create a fresh workspace, you have to re-do these settings. You can create a new workspace with copied preferences by selecting File \u2023 Switch workspace \u2023 Other... , and then checking Preferences under Copy settings . Now that Eclipse is set up, continue with creating a language project","title":"Installing the Spoofax 3 language workbench"},{"location":"tutorial/install/#installing-the-spoofax-3-language-workbench","text":"This tutorial gets you set up for language development in Spoofax 3 by installing the Spoofax 3 Eclipse LWB environment.","title":"Installing the Spoofax 3 language workbench"},{"location":"tutorial/install/#requirements","text":"Spoofax 3 runs on the major operating systems: Windows (64 bits) macOS (64 bits) Linux (64 bits) Other than that, nothing is required as everything is contained in the archive we are going to download.","title":"Requirements"},{"location":"tutorial/install/#download","text":"To get started, we will download a premade Eclipse installation that comes bundled with the Spoofax 3 LWB plugin. We will download version 0.22.0 released on 27-05-2024. Download the latest version for your platform: Windows 64-bit with embedded JVM macOS 64-bit with embedded JVM Linux 64-bit with embedded JVM These are bundled with an embedded JVM so that you do not need to have a JVM installed. If your system has a JVM of version 17 (exactly, higher versions are not supported), and would rather use that, use the following download links instead: Windows 64-bit macOS 64-bit Linux 64-bit","title":"Download"},{"location":"tutorial/install/#unpack","text":"Unpack the downloaded archive to a directory with write access . Write access is required because Eclipse needs to write to several configuration files inside its installation. Warning The unpacked directory or application may be renamed, but do not include spaces or other characters that would not be allowed in a URI (i.e., : ? # [ ] @ ). The same is true for the directory the archive is extracted to. Failing to do so breaks a built-in classpath detection mechanism which will cause Java compilation errors. Warning On Windows do not unpack the Eclipse installation into Program Files , because no write access is granted there, breaking both Eclipse and Spoofax. On macOS Sierra (10.12) and above, after unpacking, open the Terminal and navigate to the directory where the Spoofax3.app file is located and execute: xattr -rc Spoofax3.app","title":"Unpack"},{"location":"tutorial/install/#running-eclipse","text":"Start up Eclipse, depending on your operating system: Windows: run Spoofax3/eclipse.exe macOS run Spoofax3.app Linux run Spoofax3/eclipse Warning macOS Sierra (10.12) and above will mark the unpacked Spoofax3.app as \"damaged\" due to a modified signed/notarized application, because we have modified the eclipse.ini file inside it. To fix this, open the Terminal, navigate to the directory where the Spoofax3.app file is located, and execute: xattr -rc Spoofax3.app After starting up, choose where your workspace will be stored. The Eclipse workspace will contain all of your settings, and is the default location for new projects. Warning Currently, there are several bugs regarding spaces in the workspace path, so ensure there are no spaces on the workspace path.","title":"Running Eclipse"},{"location":"tutorial/install/#configuring-eclipses-preferences","text":"Some Eclipse preferences unfortunately have sub-optimal defaults. After you have chosen a workspace and Eclipse has completely started up (and you have closed the Welcome page), go to the Eclipse preferences by pressing Cmd + , on macOS and by going to the Window \u2023 Preferences menu on others, and set these options: General \u2023 Startup and Shutdown Enable: Refresh workspace on startup General \u2023 Workspace Enable: Refresh using native hooks or polling General \u2023 Workspace \u2023 Build Enable: Save automatically before manual build We need to make sure that Eclipse has detected an installed JRE . Open the Eclipse preferences and go to the Java \u2023 Installed JREs page: If there are no installed JREs, and you've downloaded an Eclipse installation with an embedded JVM , press Search... and navigate to the location where you unpacked the Eclipse installation, and choose the jvm directory in it. Then press the checkmark of the JRE to activate it. If there are no installed JREs, and you've downloaded an Eclipse installation without an embedded JVM , press Search... and navigate to the location where your JVM installed, and choose it. Then press the checkmark of the JRE to activate it. If there are one or more installed JVMs, but none are selected, select an appropriate one by pressing the checkmark. If there are one or more installed JVMs, and one is selected, you are good to go. Finally, you may configure Eclipse to your liking. Some typical settings to adjust: General \u2023 Editors \u2023 Text Editors Displayed tab width : change to your desired tab width. Most of Spoofax uses 2 by convention. Insert spaces for tabs : enable, as Spoofax uses spaces by convention. Show print margin : enable and set if you want to set a maximum line length. Show line numbers : enable if you want to see line numbers. General \u2023 Appearance : choose a Theme to your liking. General \u2023 Appearance \u2023 Colors and Fons \u2023 Text Font : the JetBrains Mono font supports ligatures and is used in (most of) the lecture slides (install separately). General \u2023 Keys : change keybindings. Tip These preferences are stored per workspace. If you create a fresh workspace, you have to re-do these settings. You can create a new workspace with copied preferences by selecting File \u2023 Switch workspace \u2023 Other... , and then checking Preferences under Copy settings . Now that Eclipse is set up, continue with creating a language project","title":"Configuring Eclipse's preferences"},{"location":"tutorial/pcf_tutorial/","text":"A Spoofax tutorial implementing Programming Computable Functions (PCF) \u00b6 In computer science, Programming Computable Functions (PCF) is a typed functional language introduced by Gordon Plotkin in 1977, based on previous unpublished material by Dana Scott. It can be considered to be an extended version of the typed lambda calculus or a simplified version of modern typed functional languages such as ML or Haskell. -- https://en.wikipedia.org/wiki/Programming_Computable_Functions With this tutorial you will hopefully be able to define the programming language PCF in the Spoofax language workbench in around one hour. In order to get started quickly, you should clone the Git repository connected to this tutorial. The template directory contains a minimal project setup that allows you to start implementing PCF. Based on this tutorial, you are hopefully able to implement the syntax and static semantics of PCF yourself. In case you get stuck, you can always peek at the example implementation in the implementation directory. Getting started \u00b6 You will need an installation of the Spoofax language workbench, you can find the latest release here . The recommended version to download is the one with embedded JVM. Note that on MacOS and Linux there are some extra instructions after unpacking. You can now import the project following the instructions on the website , which come down to File > Import... ; General > Existing Projects into Workspace ; Next > ; tick Select root directory: ; Browse... to the directory with the spoofaxc.cfg and press Open ; Finish . Now you should Project > Build Project after selecting the project in the Package Explorer . Within the project you will be working on files in the src and test directories. Source material for PCF \u00b6 Since the original publications on PCF were more concerned with semantics than syntax, it is a bit difficult to trace the syntax of PCF. We will use the definition of PCF from a book by John C. Mitchell called \"Foundation for Programming Languages\" (Mitchell 1996). Note that you might also find a definition by Dowek and Levy (Dowek et al. 2011) called Mini-ML, or PCF, this is not the version that we will use here. Chapter 2 of Foundation for Programming Languages about the language PCF is freely available . Grammar \u00b6 This is a slightly massaged grammar of so-called \"pure pcf\" from section 2.2.6: e ::= x (variable reference) | if e then e else e (if condition) | \\x : t. e (function abstraction) | e e (function application) | fix e (fixed point) | true | false (boolean constants) | Eq? e e (equality check) | n (natural number constant) | e + e (arithmetic operations) | (e) (parenthesised expression) t ::= nat (natural number type) | bool (boolean type) | t -> t (function type) | (t) (parenthesised type) As you can see, PCF is a small functional programming language. It is an expression based language where e is the expression sort, and t is the type sort. There are also lexical sorts in this grammar, namely x for names, and n for numeric constants. The other words and symbols are keywords and operators. We've replaced some of the non-ASCII notation from the book into an ASCII version to make it easier to type, but if you have a good input method for non-ascii symbols, feel free to use those in your grammar. We've also added parentheses to both sorts for grouping. We have excluded pairs from the grammar, to make the language a little smaller and hopefully make this tutorial completable in one hour. The grammar in the book is more type-directed, which constrains what programs the parser can parse to already be closer to the set of programs that are actually typed and therefore in the language. While this might seem advantageous, in practice it is nicer for a user to have a wide range of programs parse and be given highlighting. The type checker can give a much clearer explanation of why a program is not acceptable than a parser based on a grammar that encodes some type information. SDF3 \u00b6 With a context-free grammar available to us, we can start implementing the syntax of PCF in Spoofax. For this we use SDF3, the Syntax Definition Formalism version 3. You'll find that the template project already has some .sdf3 files ready for you: lex.sdf3 , expr.sdf , type.sdf3 and main.sdf3 . The main file defines the start symbol of the grammar which is used by the editor to know where to start parsing. Expressions go in expr.sdf3 , types in type.sdf3 and we have already provided you with a lexical syntax in lex.sdf3 . Have a look, you will find the definition of lexical sorts Name , Number and Keyword , where Name and Number are defined as regular expressions, and Keyword is defined as a few options of literal strings that correspond with keywords from the grammar. Then Name is restricted by a rejection rule to not match anything that can be parsed as Keyword . The lexical restrictions make sure that names and numbers are matched greedily. Aside: Grammars in SDF3 define both lexical and context-free syntax, both of which are handled together by a character-level parsing algorithm called SGLR. This makes it hard to provide truly greedy regular expression by default, and instead we express that a name is not allowed to be directly followed by a letter or number, which can be handled better by a parser and still makes things greedy. At the end of the file you find a defining of LAYOUT , which is the white space (and possibly comments) that are allowed between parts of the context-free syntax that we'll specify together for expressions and types. In the files expr.sdf3 and type.sdf3 you will find the sort definitions for expressions and types with some but not all rules. As you can see, there is template syntax in SDF3 which is both a convenient way to write your syntax and a hint for how it might be formatted in a program. Exercise. Try writing the remaining part of the grammar yourself. Once you've built the project again, you can try out the newly added parts of a grammar. In test/test.spt file you will find test written in the SPoofax Testing language SPT . In this special language workbench testing language we can test many things on a high level. For now we can more parse succeeds and parse fails tests and see failing tests get an error marker in the editor immediately. You can also with a parse to test where you can specify the abstract syntax tree you expect. Of course you can also write your PCF programming language in its own editor. There is already an example.pcf file, where you can see the syntax highlighting derived from your grammar. You can also view the abstract syntax tree of this program through the menu Spoofax > Debug > Show parsed AST . The (continuous) version even updates as you update your program. You might find that writing a program 1 + 1 + 1 fails both expectations, because it is in our PCF language, but the parsing isn't entirely successful . Instead the result, which you can also write as an expectation, is parse ambiguous . We need to specify in the grammar what the associativity of the program is, whether it's (1 + 1) + 1 or 1 + (1 + 1) . Let's pick the former and use the {left} annotation on the Expr.Add rule. Now your double-add test should work. In fact we can now use the parse to test to specify that we expect Add(Add(Num(\"1\"), Num(\"1\")), Num(\"1\")) . That is a little cumbersome to write though. What we can also do is write another program between double brackets: [[(1 + 1) + 1]] . Because the round brackets are not in the AST, this comes down to the same test. Now that we're familiar with ambiguities and testing for them, we should root out the other ones in our grammar. You'll find that most grammar productions in PCF are not ambiguous with themselves, but mostly with each other. This is a priority problem, which is specified in a context-free priorities section of the grammar. You can write Expr.App > Expr.Add to specify that application binds tighter than addition. You can write out pairs of these with commas in between, or a longer chain of > , which is more common and is a reminder that priority is transitive. You can also make groups of expressions of the same priority, like { Expr.If Expr.Lam Expr.Fix } . Exercise. See if you can figure out a good set of priorities and write some tests for them. You can check your list against ours in the implementation . Static Semantics \u00b6 PCF has a relatively simple type system, defined as follows: ---------------- [T-True] \u0393 |- true : bool ----------------- [T-False] \u0393 |- false : bool ------------ [T-Nat] \u0393 |- n : nat \u0393 |- e1 : nat \u0393 |- e2 : nat ------------------ [T-Add] \u0393 |- e1 + e2 : nat \u0393 |- e1 : t \u0393 |- e2 : t --------------------- [T-Eq] \u0393 |- Eq? e1 e2 : bool \u0393 |- e : bool \u0393 |- e1 : t \u0393 |- e2 : t ----------------------------- [T-If] \u0393 |- if e then e1 else e2 : t (x, t) \u03f5 \u0393 ---------- [T-Var] \u0393 |- x : t (x, t); \u0393 |- e : t' ------------------------ [T-Abs] \u0393 |- \\x : t. e : t -> t' \u0393 |- e1 : t' -> t \u0393 |- e2 : t' ----------------- [T-App] \u0393 |- e1 e2 : t \u0393 |- e : t -> t --------------- [T-Fix] \u0393 |- fix e : t In this type system, all constructs have monomorphic types. The equality operator accepts any operands, as long as the types of the left and right operand are equal. Similarly, the types of the then-branch and the else-branch of an if-expression should be equal. Variables are typed in a typing environment \u0393, which is extended by lambda abstractions. This is all similar to the regular definition of the lambda calculus (see e.g. Pierce (2002), chap. 9). The typing rule for the fixpoint operator ensures its argument is an endofunction, and types it with the (co)domain of the function. Statix \u00b6 Given this type system and a parser derived from the syntax specification, we can start defining our type-checker in Spoofax. We use the Statix meta-language for type system specification for this. In Statix, type-systems can be expressed in a declarative style, closely related to formal inference rules, such as given above. However, instead of typing environments, scope graphs are used for name binding. (Scope Graphs will be explained in more detail later in this tutorial.) The backend (often referred to as 'solver') interprets the specification applied to an AST of the object language (PCF in this case) as a constraint program, which yields an executable type checker. So let's get started. We will define our type-system in the src/expr.stx file. This file already imports expr-sig and type-sig , which makes the abstract syntax of the language, which is derived from the syntax definition, available. In addition, there is a declaration for a user-defined constraint typeOfExpr , which has type scope * Expr -> Type . That means that the constraint accepts a scope and an expression, and returns a type for it. This can be read as \u0393 |- e : t , where the scope argument takes the role of the environment \u0393. To test the type system, it is recommended to open the example.pcf and test/test.spt files as well. A lonely constraint declaration is useless: there are no ways a typeOfExpr constraint can be solved. We need to add rules for this constraint. Rules can be compared to cases in a functional language or, even better, inference rules as given above. Lets look at an example: typeOfExpr(_, True()) = Bool(). This rule states the simple fact that a true constant has type bool . Do you see the similarity with our T-True rule? Exercise: Define the T-False and T-Nat rules in your Statix specification. That was not too hard. However, not all rules are that simple. Some of them have premises . In Statix, we encode them after a turnstile symbol :- . For example: typeOfExpr(s, Eq(e1, e2)) = Bool() :- {T} typeOfExpr(s, e1) == T, typeOfExpr(s, e2) == T. This rule is a bit more complicated, so lets break it down part by part. The meaning of the rule head ( typeOfExpr(s, Eq(e1, e2)) = Bool() ) should be familiar by now. An equality comparison in scope s has type Bool() . Then, after the turnstile, there is an existential constraint {T} , which introduced the unification variable T . One might read it as \u2203 T. ... . Then there are two premises of the rule that assert that e1 and e2 have the type T . As a unification variable can only have a single value, this effectively enforces both operands to have the same type. Exercise: Define the T-Add and T-If rules. It is also possible to use SPT to test your type system. To apply the type-checker in a test, use the analysis succeeds and analysis fails expectations. For example: test cannot compare nat and bool [[ Eq? true 42 ]] analysis fails Exercise: Define some SPT tests that cover all currently typed constructs. Scope Graphs \u00b6 Now it is time to look add lambda abstractions and applications. In Statix, typing these constructs uses scope graphs. We will explore how scope graphs work using some example programs. The rule for lambda abstraction is already defined as follows: typeOfExpr(s, Lam(x, T, e)) = Fun(T, T') :- {s_lam} new s_lam, s_lam -P-> s, !var[x, T] in s_lam, typeOfExpr(s_lam, e) == T'. In this rule, a lambda expression gets the type Fun(T, T') . The input type T is explicitly provided by the syntax, while the result type T' is inferred from the lambda body. However, as can be seen in the T-Abs rule, the body is typed in an extended context . In Statix, that is encoded by the first three constraints. The new s_lam constraint generates a fresh node in the scope graph, and binds a reference to that node to the unification variable s_lam . That scope encodes the context of the body. To indicate that s_lam inherits all declarations from its parent context (\u0393 in the rule), the s_lam -P-> s constraints asserts an edge from s_lam to s . This ensures that a query in s_lam (explained later) can reach declarations in s . The edge is labeled with an edge label P . In Statix all edges are labeled, and labels have to be introduced explicitly. In the stub, the label P was already predefined in the signature section above the rule for abstractions. Finally, we need to extend the s_lam context with our new variable. This is what the !var[x, T] in s_lam constraint does. This constraint, which is similar to the (x, t); in T-Abs , creates a declaration with name x and type T in s_lam . A declaration always uses a relation , which is var in this case. The var relation is also declared in the signature section above. To aid debugging, scope graph can be inspected. For example, open an example.pcf file, and add this program: (\\double: nat -> nat. double 21) \\x: nat. x + x Now, open the menu Spoofax > Debug > Show formatted scope graph (continuous) . A new window should be opened with (approximately) this content: scope graph #-s_lam_20-4 { relations { expr!var : (\"double\", Fun(Nat(), Nat())) } edges { expr!P : #-s_glob_1-5 } } #-s_lam_10-2 { relations { expr!var : (\"x\", Nat()) } edges { expr!P : #-s_glob_1-5 } } This file shows two scopes: #-s_lam_20-4 and #-s_lam_10-2 . These scopes correspond to the bodies of the first and second abstraction, respectively. Both scopes have a single declaration, shown in the relations block. The contents of the relations should not be surprising: they are the declarations we asserted using our !var[x, T] in s_lam constraint! Similarly, there are two P -labeled edges to #s_glob_1-5 . This scope is the global scope, which does not have its own entry because it is empty. In fact, it corresponds to the empty top-level environment a regular typing derivation would start with. Now, we have seen how to extend a context, but how should we read it? Reading a context corresponds to doing a query in Statix. In PCF, the only time we use a query is when resolving a variable in the T-Var rule. In Statix, this rule is defined as follows: typeOfExpr(s, Var(x)) = T :- {Ts} query var filter P* and { x' :- x' == x } min $ < P in s |-> Ts, referenceTypeOk(x, T, Ts). We ignore the referenceTypeOk constraint for now, as it is only there to ensure the error messages are easier to interpret. The interesting part is the query constraint. This constraint takes quite some parameters, which we will analyse one by one. First, there is the var parameter, which is the relation to query. In languages that, unlike PCF, have multiple relations, this argument ensures that the query will only find declarations under the var relation, such as the ones asserted by the T-Abs rule we've discussed earlier. Second, there is the P* argument. This is a regular expression that describes valid paths in the scope graph. In this case, it ensures that the query can resolve in the local context, and all parent contexts. Exercise. Change P* into P+ and test the program \\x: nat. x . Does it type correctly? Why (not)? Third, there is the data well-formedness condition { x' :- x' == x } . This is an anonymous unary predicate that compares the name of a declaration (bound to x' ) to the reference ( x ). Only declarations for which the constraint holds (i.e., x' == x can be solved) are returned in the query answer. This excludes reachable declarations with the wrong name. Exercise. Change { x' :- x' == x } to { x' :- true } and test the program \\x: nat. \\y: bool. x + 1 . Does it type correctly? Why (not)? Fourth, there is the $ < P argument. This argument indicates that the end-of-path label $ binds closer than the P label. This means that shorter paths are preferred over longer paths, essentially modelling shadowing. This is best seen in action: Exercise. Remove $ < P and test the program \\x: bool. \\x: nat. x + 1 . Does it type correctly? Why (not)? Exercise. Add P < $ and test the program \\x: bool. \\x: nat. x + 1 . Does it type correctly? Why (not)? Exercise: Define the T-App and T-Fix rules. Congratulations! You have now defined a fully functional frontend for PCF. A Small Detour on Editor Services. \u00b6 Type information is often used for editor services and transformations. This is tightly integrated in the Spoofax language as well. For example, hover your mouse over a variable reference for a moment. After some time, a tooltip with (e.g.) the text Type: Nat() will show up. Or, even fancier, CTRL-Click (Cmd-Click on Mac) on a reference. Your cursor now should jump to the binder that introduced the variable. To understand this behaviour, we have to look into the second referenceTypeOk rule. This rule contains the following constraints: @x.type := T, @x.ref := x' In this rule x is the reference, and T is the expected type. The constraint @x.type := T sets the value of the type property of x to T . The Spoofax editor will read such type properties and display them in a tooltip. That explains the tooltip we observed earlier. In addition, the x' variable is the name of the declaration. Unobservable to the user, this name has its position attached. By assigning it as the ref property of the reference x , the editor reference resolution knew where to put the cursor when x was CTRL-clicked! Syntactic Sugar \u00b6 These are the syntactic extensions (syntactic sugar) found in the book we're following: e ::= ... | let x : t = e in e (let binding) | let x(x : t) : t = e in e (let function binding) | letrec x : t = e in e (let recursive binding) | letrec x(x : t) : t = e in e (let recursive binding) These extensions are defined as sugar, in that you can transform them into an equivalent program using only the \"pure pcf\" syntax. That's exactly how we will also implement this sugar. Let's first summarise the meaning of these extensions: let x : t = e1 in e2 == (\\x : t. e2) e1 let x1(x2 : t1) : t2 = e1 in e2 == let x1 : t1 -> t2 = \\x2 : t1. e1 in e2 == (\\x1 : t1 -> t2. e2) (\\x2 : t1. e1) letrec x : t = e1 in e2 == let x : t = fix \\x : t. e1 in e2 == (x : t. e2) (fix \\x : t. e1) letrec x1(x2 : t1) : t2 = e1 in e2 == letrec x1 : t1 -> t2 = \\x2 : t1. e1 in e2 == let x1 : t1 -> t2 = fix \\x1 : t1 -> t2. \\x2 : t1. e1 in e2 == (\\x1 : t1 -> t2. e2) (fix \\x1 : t1 -> t2. \\x2 : t1. e1) As you can see, we can use lambdas for binding variables in a let , and we can use fix for recursive definitions in letrec . Function definition syntax in let is of course also just sugar for lambdas. Stratego 2 \u00b6 Let's turn these equations into executable code. We will use the term-writing language Stratego 2 for this task. You can find code of this language in .str2 files such as main.str2 and desugar.str2 . In the latter you will find a prewritten strategy desugar , defined as a type preserving transformation ( TP ) that traverses topdown and tries to apply the desugar-let rewrite rules. In the rewrite rules is desugar-let , also type preserving, but it currently fails. You can see an example rewrite rule that turns let bound functions into the applications of lambdas from our equations above, but this uses the LetF abstract syntax, which we have not defined yet. So before we start writing our rewrite rules, we should first define the appropriate syntax in expr.sdf3 . Exercise: Define syntax for let and letrec , and don't forget the priority rules of the newly added syntax. With the syntax defined, we can now start writing our rewrite rules. As suggested by the example one for let-bound functions, we are writing the abstract syntax pattern of our sugar programs with variables, then an arrow -> , and then the abstract syntax pattern of our final result. This way we only have to traverse our program once to apply the rules. We could go topdown or bottomup with this approach, either direction works. Exercise: Write the remaining rewrite rules. We could also write our desugaring differently, in the smaller steps of the equation, by repeatedly applying rewrite rules. Our strategy would then be outermost or innermost , and we could rewrite our Let as usual, but write our Let(Rec)F to Let(Rec) and our LetRec to Let with Fix . Exercise: Write the single step rewrite rules. (This is demonstrated in desugar2 of the example implementation .) You can factor out the similarity of the function-binding let and letrec to their normal counterpart. Bonus Exercise: Define new constructors for Let and LetF that take an extra argument that marks if it is a normal or letrec version. Define a separate desugaring step to transform your program into these constructors. Now define the single step rewrite rules from before, combining the similar one into a single rule. (This is demonstrated in desugar3 of the example implementation .) With your desugaring now complete, you can add a call to desugar to pre-analyze in src/main.str2 . This will eliminate the let related constructors before you analyse your program, so you do not need to extend your Statix specification. References \u00b6 (Mitchell 1996) Mitchell, John C. (1996). The Language PCF. In: Foundations for Programming Languages. https://theory.stanford.edu/~jcm/books/fpl-chap2.ps (Dowek et al. 2011) Dowek, G., L\u00e9vy, JJ. (2011). The Language PCF. In: Introduction to the Theory of Programming Languages. Undergraduate Topics in Computer Science. Springer, London. https://doi.org/10.1007/978-0-85729-076-2_2 (Pierce 2002) Pierce, Benjamin C. (2002). Types and Programming Languages. MIT Press, Cambridge, Massachusetts.","title":"A Spoofax tutorial implementing Programming Computable Functions (PCF)"},{"location":"tutorial/pcf_tutorial/#a-spoofax-tutorial-implementing-programming-computable-functions-pcf","text":"In computer science, Programming Computable Functions (PCF) is a typed functional language introduced by Gordon Plotkin in 1977, based on previous unpublished material by Dana Scott. It can be considered to be an extended version of the typed lambda calculus or a simplified version of modern typed functional languages such as ML or Haskell. -- https://en.wikipedia.org/wiki/Programming_Computable_Functions With this tutorial you will hopefully be able to define the programming language PCF in the Spoofax language workbench in around one hour. In order to get started quickly, you should clone the Git repository connected to this tutorial. The template directory contains a minimal project setup that allows you to start implementing PCF. Based on this tutorial, you are hopefully able to implement the syntax and static semantics of PCF yourself. In case you get stuck, you can always peek at the example implementation in the implementation directory.","title":"A Spoofax tutorial implementing Programming Computable Functions (PCF)"},{"location":"tutorial/pcf_tutorial/#getting-started","text":"You will need an installation of the Spoofax language workbench, you can find the latest release here . The recommended version to download is the one with embedded JVM. Note that on MacOS and Linux there are some extra instructions after unpacking. You can now import the project following the instructions on the website , which come down to File > Import... ; General > Existing Projects into Workspace ; Next > ; tick Select root directory: ; Browse... to the directory with the spoofaxc.cfg and press Open ; Finish . Now you should Project > Build Project after selecting the project in the Package Explorer . Within the project you will be working on files in the src and test directories.","title":"Getting started"},{"location":"tutorial/pcf_tutorial/#source-material-for-pcf","text":"Since the original publications on PCF were more concerned with semantics than syntax, it is a bit difficult to trace the syntax of PCF. We will use the definition of PCF from a book by John C. Mitchell called \"Foundation for Programming Languages\" (Mitchell 1996). Note that you might also find a definition by Dowek and Levy (Dowek et al. 2011) called Mini-ML, or PCF, this is not the version that we will use here. Chapter 2 of Foundation for Programming Languages about the language PCF is freely available .","title":"Source material for PCF"},{"location":"tutorial/pcf_tutorial/#grammar","text":"This is a slightly massaged grammar of so-called \"pure pcf\" from section 2.2.6: e ::= x (variable reference) | if e then e else e (if condition) | \\x : t. e (function abstraction) | e e (function application) | fix e (fixed point) | true | false (boolean constants) | Eq? e e (equality check) | n (natural number constant) | e + e (arithmetic operations) | (e) (parenthesised expression) t ::= nat (natural number type) | bool (boolean type) | t -> t (function type) | (t) (parenthesised type) As you can see, PCF is a small functional programming language. It is an expression based language where e is the expression sort, and t is the type sort. There are also lexical sorts in this grammar, namely x for names, and n for numeric constants. The other words and symbols are keywords and operators. We've replaced some of the non-ASCII notation from the book into an ASCII version to make it easier to type, but if you have a good input method for non-ascii symbols, feel free to use those in your grammar. We've also added parentheses to both sorts for grouping. We have excluded pairs from the grammar, to make the language a little smaller and hopefully make this tutorial completable in one hour. The grammar in the book is more type-directed, which constrains what programs the parser can parse to already be closer to the set of programs that are actually typed and therefore in the language. While this might seem advantageous, in practice it is nicer for a user to have a wide range of programs parse and be given highlighting. The type checker can give a much clearer explanation of why a program is not acceptable than a parser based on a grammar that encodes some type information.","title":"Grammar"},{"location":"tutorial/pcf_tutorial/#sdf3","text":"With a context-free grammar available to us, we can start implementing the syntax of PCF in Spoofax. For this we use SDF3, the Syntax Definition Formalism version 3. You'll find that the template project already has some .sdf3 files ready for you: lex.sdf3 , expr.sdf , type.sdf3 and main.sdf3 . The main file defines the start symbol of the grammar which is used by the editor to know where to start parsing. Expressions go in expr.sdf3 , types in type.sdf3 and we have already provided you with a lexical syntax in lex.sdf3 . Have a look, you will find the definition of lexical sorts Name , Number and Keyword , where Name and Number are defined as regular expressions, and Keyword is defined as a few options of literal strings that correspond with keywords from the grammar. Then Name is restricted by a rejection rule to not match anything that can be parsed as Keyword . The lexical restrictions make sure that names and numbers are matched greedily. Aside: Grammars in SDF3 define both lexical and context-free syntax, both of which are handled together by a character-level parsing algorithm called SGLR. This makes it hard to provide truly greedy regular expression by default, and instead we express that a name is not allowed to be directly followed by a letter or number, which can be handled better by a parser and still makes things greedy. At the end of the file you find a defining of LAYOUT , which is the white space (and possibly comments) that are allowed between parts of the context-free syntax that we'll specify together for expressions and types. In the files expr.sdf3 and type.sdf3 you will find the sort definitions for expressions and types with some but not all rules. As you can see, there is template syntax in SDF3 which is both a convenient way to write your syntax and a hint for how it might be formatted in a program. Exercise. Try writing the remaining part of the grammar yourself. Once you've built the project again, you can try out the newly added parts of a grammar. In test/test.spt file you will find test written in the SPoofax Testing language SPT . In this special language workbench testing language we can test many things on a high level. For now we can more parse succeeds and parse fails tests and see failing tests get an error marker in the editor immediately. You can also with a parse to test where you can specify the abstract syntax tree you expect. Of course you can also write your PCF programming language in its own editor. There is already an example.pcf file, where you can see the syntax highlighting derived from your grammar. You can also view the abstract syntax tree of this program through the menu Spoofax > Debug > Show parsed AST . The (continuous) version even updates as you update your program. You might find that writing a program 1 + 1 + 1 fails both expectations, because it is in our PCF language, but the parsing isn't entirely successful . Instead the result, which you can also write as an expectation, is parse ambiguous . We need to specify in the grammar what the associativity of the program is, whether it's (1 + 1) + 1 or 1 + (1 + 1) . Let's pick the former and use the {left} annotation on the Expr.Add rule. Now your double-add test should work. In fact we can now use the parse to test to specify that we expect Add(Add(Num(\"1\"), Num(\"1\")), Num(\"1\")) . That is a little cumbersome to write though. What we can also do is write another program between double brackets: [[(1 + 1) + 1]] . Because the round brackets are not in the AST, this comes down to the same test. Now that we're familiar with ambiguities and testing for them, we should root out the other ones in our grammar. You'll find that most grammar productions in PCF are not ambiguous with themselves, but mostly with each other. This is a priority problem, which is specified in a context-free priorities section of the grammar. You can write Expr.App > Expr.Add to specify that application binds tighter than addition. You can write out pairs of these with commas in between, or a longer chain of > , which is more common and is a reminder that priority is transitive. You can also make groups of expressions of the same priority, like { Expr.If Expr.Lam Expr.Fix } . Exercise. See if you can figure out a good set of priorities and write some tests for them. You can check your list against ours in the implementation .","title":"SDF3"},{"location":"tutorial/pcf_tutorial/#static-semantics","text":"PCF has a relatively simple type system, defined as follows: ---------------- [T-True] \u0393 |- true : bool ----------------- [T-False] \u0393 |- false : bool ------------ [T-Nat] \u0393 |- n : nat \u0393 |- e1 : nat \u0393 |- e2 : nat ------------------ [T-Add] \u0393 |- e1 + e2 : nat \u0393 |- e1 : t \u0393 |- e2 : t --------------------- [T-Eq] \u0393 |- Eq? e1 e2 : bool \u0393 |- e : bool \u0393 |- e1 : t \u0393 |- e2 : t ----------------------------- [T-If] \u0393 |- if e then e1 else e2 : t (x, t) \u03f5 \u0393 ---------- [T-Var] \u0393 |- x : t (x, t); \u0393 |- e : t' ------------------------ [T-Abs] \u0393 |- \\x : t. e : t -> t' \u0393 |- e1 : t' -> t \u0393 |- e2 : t' ----------------- [T-App] \u0393 |- e1 e2 : t \u0393 |- e : t -> t --------------- [T-Fix] \u0393 |- fix e : t In this type system, all constructs have monomorphic types. The equality operator accepts any operands, as long as the types of the left and right operand are equal. Similarly, the types of the then-branch and the else-branch of an if-expression should be equal. Variables are typed in a typing environment \u0393, which is extended by lambda abstractions. This is all similar to the regular definition of the lambda calculus (see e.g. Pierce (2002), chap. 9). The typing rule for the fixpoint operator ensures its argument is an endofunction, and types it with the (co)domain of the function.","title":"Static Semantics"},{"location":"tutorial/pcf_tutorial/#statix","text":"Given this type system and a parser derived from the syntax specification, we can start defining our type-checker in Spoofax. We use the Statix meta-language for type system specification for this. In Statix, type-systems can be expressed in a declarative style, closely related to formal inference rules, such as given above. However, instead of typing environments, scope graphs are used for name binding. (Scope Graphs will be explained in more detail later in this tutorial.) The backend (often referred to as 'solver') interprets the specification applied to an AST of the object language (PCF in this case) as a constraint program, which yields an executable type checker. So let's get started. We will define our type-system in the src/expr.stx file. This file already imports expr-sig and type-sig , which makes the abstract syntax of the language, which is derived from the syntax definition, available. In addition, there is a declaration for a user-defined constraint typeOfExpr , which has type scope * Expr -> Type . That means that the constraint accepts a scope and an expression, and returns a type for it. This can be read as \u0393 |- e : t , where the scope argument takes the role of the environment \u0393. To test the type system, it is recommended to open the example.pcf and test/test.spt files as well. A lonely constraint declaration is useless: there are no ways a typeOfExpr constraint can be solved. We need to add rules for this constraint. Rules can be compared to cases in a functional language or, even better, inference rules as given above. Lets look at an example: typeOfExpr(_, True()) = Bool(). This rule states the simple fact that a true constant has type bool . Do you see the similarity with our T-True rule? Exercise: Define the T-False and T-Nat rules in your Statix specification. That was not too hard. However, not all rules are that simple. Some of them have premises . In Statix, we encode them after a turnstile symbol :- . For example: typeOfExpr(s, Eq(e1, e2)) = Bool() :- {T} typeOfExpr(s, e1) == T, typeOfExpr(s, e2) == T. This rule is a bit more complicated, so lets break it down part by part. The meaning of the rule head ( typeOfExpr(s, Eq(e1, e2)) = Bool() ) should be familiar by now. An equality comparison in scope s has type Bool() . Then, after the turnstile, there is an existential constraint {T} , which introduced the unification variable T . One might read it as \u2203 T. ... . Then there are two premises of the rule that assert that e1 and e2 have the type T . As a unification variable can only have a single value, this effectively enforces both operands to have the same type. Exercise: Define the T-Add and T-If rules. It is also possible to use SPT to test your type system. To apply the type-checker in a test, use the analysis succeeds and analysis fails expectations. For example: test cannot compare nat and bool [[ Eq? true 42 ]] analysis fails Exercise: Define some SPT tests that cover all currently typed constructs.","title":"Statix"},{"location":"tutorial/pcf_tutorial/#scope-graphs","text":"Now it is time to look add lambda abstractions and applications. In Statix, typing these constructs uses scope graphs. We will explore how scope graphs work using some example programs. The rule for lambda abstraction is already defined as follows: typeOfExpr(s, Lam(x, T, e)) = Fun(T, T') :- {s_lam} new s_lam, s_lam -P-> s, !var[x, T] in s_lam, typeOfExpr(s_lam, e) == T'. In this rule, a lambda expression gets the type Fun(T, T') . The input type T is explicitly provided by the syntax, while the result type T' is inferred from the lambda body. However, as can be seen in the T-Abs rule, the body is typed in an extended context . In Statix, that is encoded by the first three constraints. The new s_lam constraint generates a fresh node in the scope graph, and binds a reference to that node to the unification variable s_lam . That scope encodes the context of the body. To indicate that s_lam inherits all declarations from its parent context (\u0393 in the rule), the s_lam -P-> s constraints asserts an edge from s_lam to s . This ensures that a query in s_lam (explained later) can reach declarations in s . The edge is labeled with an edge label P . In Statix all edges are labeled, and labels have to be introduced explicitly. In the stub, the label P was already predefined in the signature section above the rule for abstractions. Finally, we need to extend the s_lam context with our new variable. This is what the !var[x, T] in s_lam constraint does. This constraint, which is similar to the (x, t); in T-Abs , creates a declaration with name x and type T in s_lam . A declaration always uses a relation , which is var in this case. The var relation is also declared in the signature section above. To aid debugging, scope graph can be inspected. For example, open an example.pcf file, and add this program: (\\double: nat -> nat. double 21) \\x: nat. x + x Now, open the menu Spoofax > Debug > Show formatted scope graph (continuous) . A new window should be opened with (approximately) this content: scope graph #-s_lam_20-4 { relations { expr!var : (\"double\", Fun(Nat(), Nat())) } edges { expr!P : #-s_glob_1-5 } } #-s_lam_10-2 { relations { expr!var : (\"x\", Nat()) } edges { expr!P : #-s_glob_1-5 } } This file shows two scopes: #-s_lam_20-4 and #-s_lam_10-2 . These scopes correspond to the bodies of the first and second abstraction, respectively. Both scopes have a single declaration, shown in the relations block. The contents of the relations should not be surprising: they are the declarations we asserted using our !var[x, T] in s_lam constraint! Similarly, there are two P -labeled edges to #s_glob_1-5 . This scope is the global scope, which does not have its own entry because it is empty. In fact, it corresponds to the empty top-level environment a regular typing derivation would start with. Now, we have seen how to extend a context, but how should we read it? Reading a context corresponds to doing a query in Statix. In PCF, the only time we use a query is when resolving a variable in the T-Var rule. In Statix, this rule is defined as follows: typeOfExpr(s, Var(x)) = T :- {Ts} query var filter P* and { x' :- x' == x } min $ < P in s |-> Ts, referenceTypeOk(x, T, Ts). We ignore the referenceTypeOk constraint for now, as it is only there to ensure the error messages are easier to interpret. The interesting part is the query constraint. This constraint takes quite some parameters, which we will analyse one by one. First, there is the var parameter, which is the relation to query. In languages that, unlike PCF, have multiple relations, this argument ensures that the query will only find declarations under the var relation, such as the ones asserted by the T-Abs rule we've discussed earlier. Second, there is the P* argument. This is a regular expression that describes valid paths in the scope graph. In this case, it ensures that the query can resolve in the local context, and all parent contexts. Exercise. Change P* into P+ and test the program \\x: nat. x . Does it type correctly? Why (not)? Third, there is the data well-formedness condition { x' :- x' == x } . This is an anonymous unary predicate that compares the name of a declaration (bound to x' ) to the reference ( x ). Only declarations for which the constraint holds (i.e., x' == x can be solved) are returned in the query answer. This excludes reachable declarations with the wrong name. Exercise. Change { x' :- x' == x } to { x' :- true } and test the program \\x: nat. \\y: bool. x + 1 . Does it type correctly? Why (not)? Fourth, there is the $ < P argument. This argument indicates that the end-of-path label $ binds closer than the P label. This means that shorter paths are preferred over longer paths, essentially modelling shadowing. This is best seen in action: Exercise. Remove $ < P and test the program \\x: bool. \\x: nat. x + 1 . Does it type correctly? Why (not)? Exercise. Add P < $ and test the program \\x: bool. \\x: nat. x + 1 . Does it type correctly? Why (not)? Exercise: Define the T-App and T-Fix rules. Congratulations! You have now defined a fully functional frontend for PCF.","title":"Scope Graphs"},{"location":"tutorial/pcf_tutorial/#a-small-detour-on-editor-services","text":"Type information is often used for editor services and transformations. This is tightly integrated in the Spoofax language as well. For example, hover your mouse over a variable reference for a moment. After some time, a tooltip with (e.g.) the text Type: Nat() will show up. Or, even fancier, CTRL-Click (Cmd-Click on Mac) on a reference. Your cursor now should jump to the binder that introduced the variable. To understand this behaviour, we have to look into the second referenceTypeOk rule. This rule contains the following constraints: @x.type := T, @x.ref := x' In this rule x is the reference, and T is the expected type. The constraint @x.type := T sets the value of the type property of x to T . The Spoofax editor will read such type properties and display them in a tooltip. That explains the tooltip we observed earlier. In addition, the x' variable is the name of the declaration. Unobservable to the user, this name has its position attached. By assigning it as the ref property of the reference x , the editor reference resolution knew where to put the cursor when x was CTRL-clicked!","title":"A Small Detour on Editor Services."},{"location":"tutorial/pcf_tutorial/#syntactic-sugar","text":"These are the syntactic extensions (syntactic sugar) found in the book we're following: e ::= ... | let x : t = e in e (let binding) | let x(x : t) : t = e in e (let function binding) | letrec x : t = e in e (let recursive binding) | letrec x(x : t) : t = e in e (let recursive binding) These extensions are defined as sugar, in that you can transform them into an equivalent program using only the \"pure pcf\" syntax. That's exactly how we will also implement this sugar. Let's first summarise the meaning of these extensions: let x : t = e1 in e2 == (\\x : t. e2) e1 let x1(x2 : t1) : t2 = e1 in e2 == let x1 : t1 -> t2 = \\x2 : t1. e1 in e2 == (\\x1 : t1 -> t2. e2) (\\x2 : t1. e1) letrec x : t = e1 in e2 == let x : t = fix \\x : t. e1 in e2 == (x : t. e2) (fix \\x : t. e1) letrec x1(x2 : t1) : t2 = e1 in e2 == letrec x1 : t1 -> t2 = \\x2 : t1. e1 in e2 == let x1 : t1 -> t2 = fix \\x1 : t1 -> t2. \\x2 : t1. e1 in e2 == (\\x1 : t1 -> t2. e2) (fix \\x1 : t1 -> t2. \\x2 : t1. e1) As you can see, we can use lambdas for binding variables in a let , and we can use fix for recursive definitions in letrec . Function definition syntax in let is of course also just sugar for lambdas.","title":"Syntactic Sugar"},{"location":"tutorial/pcf_tutorial/#stratego-2","text":"Let's turn these equations into executable code. We will use the term-writing language Stratego 2 for this task. You can find code of this language in .str2 files such as main.str2 and desugar.str2 . In the latter you will find a prewritten strategy desugar , defined as a type preserving transformation ( TP ) that traverses topdown and tries to apply the desugar-let rewrite rules. In the rewrite rules is desugar-let , also type preserving, but it currently fails. You can see an example rewrite rule that turns let bound functions into the applications of lambdas from our equations above, but this uses the LetF abstract syntax, which we have not defined yet. So before we start writing our rewrite rules, we should first define the appropriate syntax in expr.sdf3 . Exercise: Define syntax for let and letrec , and don't forget the priority rules of the newly added syntax. With the syntax defined, we can now start writing our rewrite rules. As suggested by the example one for let-bound functions, we are writing the abstract syntax pattern of our sugar programs with variables, then an arrow -> , and then the abstract syntax pattern of our final result. This way we only have to traverse our program once to apply the rules. We could go topdown or bottomup with this approach, either direction works. Exercise: Write the remaining rewrite rules. We could also write our desugaring differently, in the smaller steps of the equation, by repeatedly applying rewrite rules. Our strategy would then be outermost or innermost , and we could rewrite our Let as usual, but write our Let(Rec)F to Let(Rec) and our LetRec to Let with Fix . Exercise: Write the single step rewrite rules. (This is demonstrated in desugar2 of the example implementation .) You can factor out the similarity of the function-binding let and letrec to their normal counterpart. Bonus Exercise: Define new constructors for Let and LetF that take an extra argument that marks if it is a normal or letrec version. Define a separate desugaring step to transform your program into these constructors. Now define the single step rewrite rules from before, combining the similar one into a single rule. (This is demonstrated in desugar3 of the example implementation .) With your desugaring now complete, you can add a call to desugar to pre-analyze in src/main.str2 . This will eliminate the let related constructors before you analyse your program, so you do not need to extend your Statix specification.","title":"Stratego 2"},{"location":"tutorial/pcf_tutorial/#references","text":"(Mitchell 1996) Mitchell, John C. (1996). The Language PCF. In: Foundations for Programming Languages. https://theory.stanford.edu/~jcm/books/fpl-chap2.ps (Dowek et al. 2011) Dowek, G., L\u00e9vy, JJ. (2011). The Language PCF. In: Introduction to the Theory of Programming Languages. Undergraduate Topics in Computer Science. Springer, London. https://doi.org/10.1007/978-0-85729-076-2_2 (Pierce 2002) Pierce, Benjamin C. (2002). Types and Programming Languages. MIT Press, Cambridge, Massachusetts.","title":"References"}]} \ No newline at end of file diff --git a/develop/sitemap.xml.gz b/develop/sitemap.xml.gz index b9018c386..9a4f76580 100644 Binary files a/develop/sitemap.xml.gz and b/develop/sitemap.xml.gz differ