The micro:bit supports the ability to log data to the internal flash memory of the micro:bit. This can be used by users of the micro:bit to store data such as those gathered from the internal sensors of the board, e.g. the light sensor or microphone.
Users can access this data by plugging their micro:bit into a computer, and accessing the virtual MY_DATA.HTM file. This isn't a 'real' file, rather, the contents of this file is mapped directly to the full flash storage of the micro:bit.
The MY_DATA.HTM file contains a very small, 2048 block header. This contains the actual HTML used to load and render the data into a visual format, allowing users to view, copy, and download their data easily. It's important that this block is as small as possible, to limit how much it is eating into the available flash storage on board the micro:bit. Therefore, the interface contains only essential functionality and limited styling and interactivity. However, the limited interface will attempt to load the full interface, which is contained within this repository. The full interface re-parses the data and loads in-place, removing the existing interface if present.
Ideally, this process should be as seamless as possible, hiding any form of flickering as elements are swapped out, and also should be as quick as possible in comparison to the limited interface.
Unlike a traditional React app, we don't control the full HTML stack. Normally, the index.html would be generated by the React tooling, with appropriate script and CSS URLs injected into the header. But in this case, we don't use any HTML files generated from the build process -- the URLs for the main bundle script and stylesheet are instead hardcoded into the offline MY_DATA.HTM source stored within the micro:bit v2 target of its CODAL firmware.
Because of this, a fixed URL for the main bundle .js and .css is required. create-react-app uses a hash in the file name of each bundle. Whilst this could be okay, since we could just hardcode this still, this would greatly limit the flexiblity of updating the interface (requiring CODAL to be adjusted for every rebuild of the interface). We therefore use react-app-rewired, which allows for configuration of the filenames of the built bundles.
The offline MY_DATA.HTM loads the main dl.js bundle. The bootstrap process begins with the index.tsx file.
The offline datalogger exposes a dl (for datalogger) global variable on the window object. This contains a load() function definition. This function is assigned as the callback to the window's onload handler, and is intended to parse and display the data from the log.
The definition for dl.load in the offline datalog also stores the raw log data in dl.raw. But because we also want to load ourself instead of the offline view, we still need to override this. The loading process is therefore as follows:
- Store a reference to
dl.loadin a variable (baseLoad) - Replace
dl.loadwith our own handler, which sets up React and removes the existing interface elements - When
dl.loadis invoked (onload), first call the offline load method to allow us to retrieve the log data and then do all of the actions listed above
Since MY_DATA.HTM is just a raw span of flash memory, the micro:bit stores the log data in an interesting way.
The first 2048 bytes of this file contain the actual HTML data, with embedded scripts and CSS data. Immediately following the HTML is the actual log data. This log data isn't actually stored within the HTML itself, such as within table elements, as JavaScript does this instead at runtime. Instead, it is stored within a trailing HTML comment, which is accessible using document.documentElement.outerHTML. The end of the 2048 bytes matches up perfectly with the start of this comment. The raw log binary data then follows.
In order to save on storage usage, there is no actual footer for the HTML following the comment. But this is okay, as basically every modern browser will accept the malformed HTML anyway and close everything for us, exploiting it greatly to our advantage.
The best way to understand this further is to make your own micro:bit program, if you can, then log some data, plug it into your computer and have a look at the source of MY_DATA.HTM.
While the online datalogger does make quite a few assumptions about the offline datalogger, it doesn't actually make any assumptions about how the data is stored, meaning this could change in the future without any changes needed in this project.
The datalogger uses a mechanism which allows users to update the interface with the latest data without having to manually refresh or reopen the page.
This is provided by the offline datalogger interface. Every 5 seconds, an invisible iframe is spawned which loads the datalogger. This iframe sends through the new data as a cross-frame message to the parent. The parent then calculates the hash of each to compare them. If they don't match, the interface will store the latest data and give the user an option to switch to this new data.
...