Skip to content

Moddable SDK 5.4.0

Latest
Latest
Compare
Choose a tag to compare
@mkellner mkellner released this 21 Jan 18:43
5.4.0
c71e59c

Moddable SDK 5.4.0 contains improvements made between December 3, 2024 and January 20, 2025. Our goal is to publish our release at the start of the month, but we missed that this month because there's so much in this one. (These notes also include following interim releases of the Moddable SDK: 5.3.1, 5.3.2, and 5.3.3.)

Latest JavaScript Standard (ES2025!) is on Embedded Devices Now

The annual update to the JavaScript language standard is on track to be published in June, but it is already fully implemented and ready for you to use on your embedded devices.

XS 16 is the latest major revision to the XS JavaScript engine. It fully supports the new features added to the language by the ES2025 standard with nearly 100% conformance, according to test262, the official JavaScript test suite.

Here's the rundown on the new ES2025 features available to developers using the latest Moddable SDK:

  • JSON Modules – Modules have always been imported as JavaScript code. The new import attributes feature opens the door to importing other kinds of modules. The first new kind of module is JSON. In the Moddable SDK, this means you can now include JSON files in the modules section of your project's manifest and access them using import. JSON modules work with preload, so they can be parsed during the build and stored in ROM, providing instant access at runtime that uses no RAM, no matter how large the JSON.
  • Float16 – Support has been added for half-precision floating-point values with the Float16Array TypedArray and getFloat16 and setFloat16 methods of DataView. The motivation for adding Float16 to the language is optimizing AI models, but the smaller data size is relevant to IoT too, especially sensor readings, which often have a limited range of values. By using less memory for the values, more data can be stored in device memory and that data can be transmitted more efficiently over BLE and the internet, saving time, bandwidth, and energy.
  • Iterator helpers – This suite of new methods brings powerful operations to JavaScript iterators, including map, filter, take, drop, flatMap, reduce, toArray, forEach, some, every, and find. The V8 team published a great introduction to these and MDN is already up-to-date.
  • Promise.try() – A new Promise helper function that brings a common developer coding pattern into the language. MDN explains it thoroughly.
  • Additional Set methods – A suite of methods has been added to the built-in Set object to perform operations across two sets. The new methods are: intersection, union, difference, symmetricDifference, isSubsetOf, isSupersetOf, isDisjointFrom. For an excellent, illustrated explanation of these new methods, see Set Composition Methods on MDN.
  • Regular Expressions – We don't use regular expressions often in embedded projects, but XS supports them so you can use them in your projects if you want. ES2025 adds two new features to regular expressions, both of which are explained extremely well by Axel Rauschmayer on his 2ality site: pattern modifiers and duplicate named capture groups.
  • Error.isError() – This new helper function isn't part of ES2025, but because it appears poised to be included in ES2026, we've included it in XS 16

We've tested these additions thoroughly with test262 to ensure conformance with the language specification and, therefore, interoperability with other JavaScript engines. We also test with testmc, the Moddable SDK's test suite to ensure compatibility with the many modules in the Moddable SDK. We have also run hundreds of hours (and still counting) of fuzz testing using Fuzzilli to ensure that the changes in XS 16 are memory safe and resilient.

ES2025 Impact on JavaScript Engine Code Size

All of these new language features would cause a significant increase in code size for any other JavaScript engine. XS is different. When building in the Moddable SDK, the build automatically removes many features of the language that it detects are unused. For example, only the new Set methods you use are included in your build; if you don't use Set at all, the entire Set object is excluded. The same is true for Float16 and Iterator Helpers. This strip feature is just one of the many ways that XS helps embedded JavaScript developers make optimal use of their resource-constrained hardware.

Improved ESP32 Debugging Connection

No tool is more important to software developers than their debugger. This release includes a complete reworking of communication for JavaScript debugging across the full ESP32 family of microcontrollers. The original implementation has evolved over the years to support more kinds of communication and more uses on more devices. The result was getting a little messy. That's been restructured, resulting in smaller, more readable, more consistent, and more maintainable code.

As a bonus, debugging over the USB JTAG connection (common on ESP32-S3 devices like Moddable Six) is now faster and more reliable. Check out the updated implementation with separate files for communication using UART, USB CDC over TinyUSB, and USB CDC over JTAG in the new esp32/lib/debugger directory.

These changes also ensure that the selected debugging communication channel is enabled for both debug and instrumentation builds and entirely disabled for release builds. This has always been the intent, but not all devices implemented it consistently.

ECMA-419 Networking for AI and Beyond

The Moddable SDK is migrating to industry standard ECMA-419 APIs everywhere we can. As part of that, we have been taking a hard look at the implementations of the ECMA-419 networking protocols, especially for use with AI cloud services. The recent flurry of "realtime" AI services that run over WebSockets uncovered a few issues, as have some TLS configuration changes by some services. Work on video streaming as part of our recent camera integration identified issues in the HTTP client and server. We've fixed all of these, as well as making improvements to eliminate buffer copies and make more optimal use of available bandwidth. Many thanks to the developers who have shared their findings – in particular, we've seen quite a few from Japan and Europe.

A little help? C compiler support for float16_t

ES2025 adds support for 16-bit floating point values based on IEEE 754 half-precision floating-point values. Unfortunately, support for float16_t is not yet universal in C compilers. This is especially true on embedded systems and Linux distributions, both of which have reasons to take their time in adopting compiler updates. To allow float16 support to work everywhere, the XS JavaScript engine can be configured at build time to use the C compiler's float16 support or to use a built-in implementation. Our preference is to use the C compiler's implementation, as it is likely better optimized for the target silicon. Because there are so many build configurations, it is impossible to test this completely. So, we ask: if you happen to run into a problem building with the latest Moddable SDK, please report it to us immediately, especially if it looks like it is related to float16 support. We want to get that resolved quickly so no one else runs into the same problem. Thanks!

On the Horizon: ESP-IDF v5.4

ESP-IDF v5.4 was released at the start of the New Year, Our initial testing shows that some changes will be required to migrate the Moddable SDK to v5.4. The current recommended ESP-IDF version remains v5.3.1. There is no plan to migrate to v5.3.2 now that v5.4 is available; however, preliminary testing of v5.3.2 did not identify any issues, so it is likely safe to use if it contains improvements your project requires.

What does Moddable think of JS0/JSSugar?

tl;dr: we really like it.

If you are the kind of person who needs to know the details, check out our "Language Evolution" blog post.

Release Details

  • XS JavaScript engine
    • XS 16 fully supports ES2025. See notes above for details.
    • Add Error.isError()
  • Devices
    • Pico
      • Uses gcc-arm-embedded
      • Pico now requires building the pioasm tool to build .pio files
      • Arducam hm01b0 camera supports 16-level (4-bit) grayscale capture
      • DigitalBank now uses the proper API to disable the IRQ
    • ESP32
      • ESP32 Camera
        • Supports 16-level grayscale capture
        • Configurable XCLK_FREQ and JPEG_QUALITY (contributed by @kitazaki) #1447
      • Debugger communication completely revised. See notes above for deatils.
    • Nordic nRF52
      • Changes to deploy process and updated bootloader for better compatibility with macOS Sequoia (15.2)
      • nrf52/xiao targets – now uses correct LED pin numbers
  • ECMA-419
    • Flash and Update (OTA) modules added. These are implementations of new modules expected in 3rd Edition.
    • TCP Socket
      • write() method returns updated writable value
    • TLS Socket
      • Support options.more in write() method
      • Use writable value returned by TCP/TLS write
    • Listener Socket
      • lwip implementation updates instrumentation socket count
    • WebSocket Client
      • Don't mask messages sent as server
      • Eliminate a buffer copy on write
      • Use writable value returned by TCP/TLS write
      • Last packet sent in a group must use continuation opcode
      • Implement "number" for format property
      • Implement target
      • Support bring-your-own buffer in read() method with "buffer" format
    • HTTP Client
      • Use writable value returned by TCP/TLS write
      • Fixes sending partial http headers
    • MQTT Client
      • Use writable value returned by TCP/TLS write
    • HTTP Server
      • Use writable value returned by TCP/TLS write
      • Fix response of unknown length (reported by @kitazaki) #1448
    • WebSocket emulation
      • Don't require URL when using attach option
    • MQTT emulation
      • Fix bug in removeEventListener()
    • FT6202 driver again accepts vendors 0 and 2
    • PulseCount
      • Experimental implementation now available on Raspberry Pi Pico
  • Modules
    • screen not required when setting-up Commodetto (reported by @stc1988) #1445
    • TLS
      • Reduce buffer copies
      • include SAN in TL certification verification (reported by @cmidgley) #1421
      • Parse signature hash algorithms list for TLS 1.2 and beyond (fixes OpenAI handshake failure)
    • Worker using compatibility options object does sanity check on arguments for compatibility with code that doesn't expect big memory devices
    • Bluetooth
      • Clean-up to remove private APIs and fields from public view
  • Documentation
    • Document BLE GATT Characteristic Properties (reported by @stc1988) #1450
  • Tools
    • mcconfig and mcrun support JSON files in the modules section of manifests
    • xsbug
      • Fix Test pane scrolling on Linux/Windows
      • Updated filtering of test262 test cases
    • Define ESP build environment variables in mcconfig to make available to manifests (suggested by @HipsterBrown) #1443
  • Testing
    • test262 and testmc on esp32/moddable_six_cdc target use big heap and stack
    • New unit tests for TLS SAN validation
  • TypeScript
    • Split ble.d.ts into pieces corresponding to the BLE modules (contributed by @stc1988)

Contact Us

If you have questions or suggestions about anything here, please reach out:

Contributors

HipsterBrown, cmidgley, and 2 other contributors
Assets 20
Loading