Transformers

In the transition from the old configuration to the new configuration for Calliope & micro:bit the concept of transformers was introduced. These are used for either altering the XML (XSLT transformers) or the AST itself (version transformers), the purposes and functionalities are detailed below.

XSLT transformers

The main function of these is transforming the incoming, potentially invalid, XML files into data consumable by our JAXB implementation. Old XMLs files may be invalid due to changes made in Blockly over the years or user manipulation. Usually, these changes are handled in Blockly itself, with various mappings and extensions done throughout. Previously, anything that touches JAXB went through Blockly beforehand, this however changed with the introduction of the Version Transformers. In order to apply transformations to the AST before returning anything to the frontend, the XMLs need to be valid right after the importing or database loading step, otherwise JAXB will fail.

Currently, there are two XSLT transformations which are applied in sequence:

• First, the mapping.xslt handles all simple renamings from one name to another.
• Second, the expansion.xslt expands or modifies elements, e.g. ensuring that all *_getSample blocks have a MODE, PORT and SLOT (and a lot more).

These should effectively be final, as they should only capture all older changes to blockly (pre transformers) and bring the XML up to date. Changes should only be necessary when the initial implementation was wrong or missing something.

The transformers are loaded once in the singleton class XsltTransformer (injected with Guice) and operate on the XML directly in the /import and /listing REST calls.

They were validated by cross checking the output with a headless Blockly version from before any new configuration changes for Calliope were applied. This should ensure that all old programs a user may have as XML, or saved in the database should work the same as before.

Version transformers

These operate on the AST and are used to transform programs between different xmlversions. With the introduction of the new configuration, changes within the AST were necessary.

Previously, a Calliope program may contain a use of the humidity sensor connected on port A1 in a programming block. With the new configuration this should have a corresponding configuration component, for consitency's sake with the name A1, which is referenced by the programming block as A1. However, the XML representation of the port A1 is 5 (as defined through Blockly), this means that on the program side the block needs to be changed to contain A1 instead of 5 and the configuration component needs to contain the information that the humidity sensor with the name A1 is connected to pin 5.

Input program block:

<block type="robSensors_humidity_getSample" id="qjhuykBiG5lJbNF79A1R" intask="true">
    <mutation mode="HUMIDITY"/>
    <field name="MODE">HUMIDITY</field>
    <field name="SENSORPORT">5</field>
    <field name="SLOT"/>
</block>

Output program block:

<block type="robSensors_humidity_getSample" id="qjhuykBiG5lJbNF79A1R" intask="true">
    <mutation mode="HUMIDITY"/>
    <field name="MODE">HUMIDITY</field>
    <field name="SENSORPORT">A1</field>
    <field name="SLOT"/>
</block>

Output config block:

<block type="robConf_humidity" id="A1" inline="false" intask="true" deletable="true" movable="true">
    <field name="NAME">A1</field>
    <field name="PIN1">5</field>
</block>

The version transformation is structured as follows (with the version two to three transformers as an example):

• AbstractTransformerWorker:
  • Receives a project and essentially deep copies the AST using an optional ITransformerVisitor for each element.
  • If no visitor is defined no copy is created.
  • Replaces the existing program AST with the copy and the optionally generated configuration.
  • Applies the new version number and only operates on the minimum and maximum versions as configured.
• Two2ThreeTransformerWorker:
  • Specifically operates on xmlversions 0.0 to 2.0 without a visitor, it just increases the version number.
  • Used for all robots except Calliope and micro:bit.
• MbedTwoToThreeTransformerWorker:
  • Additionally uses the MbedTwo2ThreeTransformerVisitor with the MbedTwo2ThreeTransformerHelper.
• ITransformerVisitor:
  • Recreates and returns each Phrase in its respective call.
  • During the recreation, modify (similar to accept, but does not ignore disabled/not in task blocks and always returns a Phrase again) is called for any expressions, to ensure every block is visited.
• IMbedTransformerVisitor:
  • Additionally adds the Mbed specific recreation visits.
• MbedTwo2ThreeTransformerVisitor:
  • Instead of just copies, the required changes from version two to three are applied to the AST here.
  • Used and thus necessary configuration blocks are also collected here.
  • Heavily uses the MbedTwo2ThreeTransformerHelper to determine names and configuration components.
• MbedTwo2ThreeTransformerHelper:
  • Contains and supplies all the mappings from a program block with block type, mode and port to a corresponding configuration block.
  • Contains modified names for the frontend based on the internal name, e.g. robConf_humidity with 5 should be called A1.
  • Contains some Callibot specific mappings.
  • If necessary, generates the desired configuration component.

In conjunction, these provide the functionality to modify or even replace blocks in the program AST, as well as creating a configuration AST from the program XML.

Incrementing xmlversion

• Changes in the XSLT do not require increasing the version, as the XSLT transformers (should) only pull very old programs up to xmlversion=2.0.
• Small, visual changes to blocks do not require a change to the version or the transformers.
• Removing or ignoring single elements or attributes should not require incrementing the version, as long as the old XML stays valid. However, it should probably be increased anyway to stay consistent.
• Adding or changing an element or attribute should require incrementing the version, old programs do not contain all necessary information.