MIDI 2.0 Universal MIDI Packet format (UMP)

[symdeb]

What is the assessment of the impact the MIDI 2.0 packet based protocol will have on Jack audio and MIDI ? Would this need a new ALSA driver/sequencer and a change of Jack2 related APIs ?
https://www.midi.org/articles-old/details-about-midi-2-0-midi-ci-profiles-and-property-exchange

[falkTX]

A bit too early to tell, barely anything exists that supports MIDI 2.0 yet.
From my understanding, there is no need to change JACK MIDI API, but we might have to do something regarding ALSA-MIDI support. First we have to wait until the ALSA does something about it though

[symdeb]

Link ALSA https://github.com/alsa-project/alsa-lib/issues Link universal packet format https://www.midi.org/articles-old/details-about-midi-2-0-midi-ci-profiles-and-property-exchange) https://www.midi.org/articles-old/midi-2-0-scope https://juce.com/discover/stories/introducing-midi-2-0 Link first MIDI 2.0 Ready product: Roland A-88MKII https://www.midi.org/articles-old/roland-announces From MIDI.ORG: MIDI 2.0 uses a Universal MIDI Packet (UMP) format for MIDI 1.0 Protocol messages and MIDI 2.0 Protocol messages. There is a Version 2 of the USB Device Class Definition for MIDI Devices in development to support the UMP format. It will require new class drivers. All of the major OS vendors are aware of the progress. I think the ALSA team is also aware. Note: ALSA team has been made aware of this, see ALSA github repo. alsa-project/alsa-lib#24

[bbouchez]

Hello there,
I am a MMA (MIDI Manufacturers Association) member from the MIDI 2.0 workgroup and I am currently working on some prototypes involving MIDI 2.0 on JACK.
I can tell that JACK is currently perfectly compatible with MIDI 2.0 UMP packets. I have performed various tests on Windows and Linux platforms and the JACK servers on these two platforms are passing UMP packets perfectly.

[symdeb]

Is that between client created ports or with physical USB devices using the raw or sequencer setting in jack ? How would that work with Windows Win32 MIDI API that lies underneath Jack in Windows since the win32 API only supports a DWORD for MIDI events.

[bbouchez]

For now, there are no "official" physical devices yet, as the standard group is still discussing about USB implementation (there is nothing officially released about USB MIDI 2.0). On my side, I am working on Ethernet based devices, which do not need system dependent drivers.
One of the tools I have created is a RTP-UMP daemon, which allows to exchange UMP packets between different physical machines.
About Windows API : the problem is that there is not yet any official API (MME on Windows or CoreMIDI on Mac) which supports UMP packets for now. So you should not expect any existing Windows software to be compatible with UMP. For now, the only way to get these messages on a Windows machine is to use JACK to circumvent the MME API limitations.

[symdeb]

Thanks. that explains it Out of curiosity just verified UMP on Jack (Linux) with client registered ports. The data messages of 4 (and 8) bytes can indeed be exchanged between the ports. However when the packet reaches the ALSA level the first byte gets removed so only a 3 byte legacy MIDI message remains and this does reach the ALSA hardware device (or software clients) correctly.
Do however get an error *** stack smashing detected ***: terminated Aborted (core dumped) when disconnecting ports & deactivating Jack. That does not occur with messages of 3 bytes.

[bbouchez]

Sounds logic. ALSA (like Win MME or Core MIDI) knows nothing about MIDI 2.0. That's why I am using RTP-UMP (based on RTP-MIDI) hardware interfaces. From software point of view, JACK's clients must be programmed specifically to recognize UMP. I plan to work on some Open Source synths later (especially for the Zynthian target)
By the way, UMP is not only 4 and 8 bytes, there are also 16 bytes messages (but they pass through JACK like the others). From what I can see, JACK works like if these messages are SYSEX but it does not check that they start with 0xF0 (which is pretty cool as it makes JACK compatible de facto with UMP °-)

[cbix]

https://github.com/atsushieno/cmidi2

might be useful for client implementations.

[jcelerier]

things are starting to move, ALSA has alsa_ump support now: https://github.com/alsa-project/alsa-lib/blob/master/include/ump.h ; CoreMIDI has it since macOS 11.0 / iOS 14 (https://developer.apple.com/documentation/coremidi/midi_services/incorporating_midi_2_into_your_apps/) and there's WIP support on windows: https://devblogs.microsoft.com/windows-music-dev/the-new-windows-midi-services-spring-2023-update/

I'm currently adding support to https://github.com/jcelerier/libremidi

[symdeb]

After further testing jack_midi_event_write does allow to send to UMP data , however jack_midi_get_event_count and jack_midi_event_get attempt to read the data as legacy as such only sending UMP seems to work with jack, not receive..

[bbouchez]

I do not use jack_midi_event_write to send UMP, I use jack_midi_event_reserve. JACK documentation says that jack_midi_event_write is a wrapper around jack_midi_event_reserve, but it may perform some extra processing which blocks messages which do not comply to MIDI 1.0 (have to check in JACK source code). With jack_midi_event_reserve, I am able to send any UMP message, as you can see on the screenshot

Here is the test code I use to send UMP blocks.

int jack_process (jack_nframes_t nframes, void* arg)
{
void* out_port_buf = jack_port_get_buffer(OutputPort, nframes);
jack_midi_data_t* Buffer;

jack_midi_clear_buffer(out_port_buf);    // Recommended to call this at the beginning of process cycle

if (DoTest1)
{    // Test MIDI 1.0
    Buffer=jack_midi_event_reserve (out_port_buf, 0, 3);
    if (Buffer!=0)
    {
        Buffer[0]=0xB0;
        Buffer[1]=0x40;
        Buffer[2]=0x7F;
    }
    DoTest1=false;
}

// Test sending 32 bits UMP message
if (DoTest2)
{
    Buffer=jack_midi_event_reserve (out_port_buf, 0, 4);
    if (Buffer!=0)
    {
        Buffer[0]=0x20;     // MT=2, group=0
        Buffer[1]=0xB0;
        Buffer[2]=0x40;
        Buffer[3]=0x7F;
    }
    DoTest2=false;
}

// Test sending 64 bits UMP message
if (DoTest3)
{
    Buffer=jack_midi_event_reserve (out_port_buf, 0, 8);
    if (Buffer!=0)
    {
        Buffer[0]=0x40;     // MT=4, group=0
        Buffer[1]=0xB0;
        Buffer[2]=0x40;
        Buffer[3]=0x00;
        Buffer[4]=0x12;     // 32 bits CC
        Buffer[5]=0x34;
        Buffer[6]=0x56;
        Buffer[7]=0x78;
    }
    DoTest3=false;
}

// Test sending 128 bits UMP message (Device Identity Notification)
if (DoTest4)
{
    Buffer=jack_midi_event_reserve (out_port_buf, 0, 16);
    if (Buffer!=0)
    {
        Buffer[0]=0xF0;     // MT=F
        Buffer[1]=0x02;     // Status=2
        Buffer[2]=0x00;
        Buffer[3]=0x00;

        Buffer[4]=0x00;
        Buffer[5]=0x00;     // SYSEX ID 1
        Buffer[6]=0x20;     // SYSEX ID 2
        Buffer[7]=0x7C;     // SYSEX ID 3  (thank you KissBox BV!!! °-) )

        Buffer[8]=0x01;     // Device family LSB
        Buffer[9]=0x00;     // Device family MSB
        Buffer[10]=0x01;    // Device model LSB
        Buffer[11]=0x00;    // Device model MSB

        Buffer[12]=0x01;    // SW revision = 1.0.0.0
        Buffer[13]=0x00;
        Buffer[14]=0x00;
        Buffer[15]=0x00;
    }
    DoTest4=false;
}

TotalFrames+=nframes;

return 0;

} // jack_process

[symdeb]

Thanks a lot, this helps. I have added a Feature Request for adding UMP support in the jack_midi_event_write call. Guess this needs to be done both in jack2 as well as in the pipewire jack emulator (?).

[bbouchez]

I have released the complete source code of my two UMP/JACK demo applications. I have provided the links in a discussion I created : #985

[cbix]

I really like the ALSA implementation allowing clients to explicitly specify legacy or UMP and taking care of transparent translation between the two according to the MIDI 2.0 spec. Wouldn't it make sense to extend the JACK API in a similar way, so legacy and MIDI 2.0 applications can interact transparently? IIUC adding a port type is not a breaking change to the ABI, or am I missing anything?

[symdeb]

ALSA sequencer API supports all the UMP features, its a great API. ALSA does a great job for all translation handling. Since jack seems to use raw MIDI, it is quite difficult to determine a port supports UMP or not and to obtain the port capabilities. So far it seems there isn't a method to identify this from the current ALSA raw port API. The other challenge is the possible complexity of non-static function blocks when port capabilities change on the fly. Not sure how far the raw API changes will impact DAWs and other future UMP apps but could likely become quite a challenge for client applications. Ofcourse, It is possible to use the ALSA sequencer without the scheduling feature

[symdeb]

jack_midi_event_write works fine. The issues is reading UMP from ports. jack_midi_get_event_count and jack_midi_event_get . I used the same calls in in that demo app, but somehow didn't work. Those may work when its between two software jack ports because there isn't ALSA in between, though my use case is reading data from an actual MIDI UMP hardware device. Need check further..~~~ if someone has a Protozoa and verify ?

[cbix]

So far it seems there isn't a method to identify this from the current ALSA raw port API.

There is, check the kernel docs I linked above:

When the MIDI 2.0 device is probed, the kernel creates a rawmidi device for each UMP Endpoint of the device. Its device name is /dev/snd/umpC*D* and different from the standard rawmidi device name /dev/snd/midiC*D* for MIDI 1.0, in order to avoid confusing the legacy applications accessing mistakenly to UMP devices.

With the important note:

Unlike the MIDI 1.0 byte stream, UMP is a 32bit packet, and the size for reading or writing the device is also aligned to 32bit (which is 4 bytes).

The 32-bit words in the UMP packet payload are always in CPU native endianness

While it may certainly work to send UMP over an 8-bit MIDI port in JACK if both ends understand UMP, in real-world scenarios it's just not as practical as having a separate 32-bit UMP port type and make JACK handle translation between the two, e.g. when a UMP port is connected to a legacy MIDI port.

The other challenge is the possible complexity of non-static function blocks when port capabilities change on the fly

I agree, sounds to me like another reason to make JACK "aware" of UMP ;)

if someone has a Protozoa and verify

Off-topic but in the past I've used a RasPi Zero 2 in USB gadget mode for this, based on the kernel docs, see https://gist.github.com/cbix/97a341c2857fd4f55d0cd19ccf6c354b

[cbix]

PipeWire 1.3.81 has been released today, including a new JACK port type for UMP in an extension header:

/* MIDI 2.0 UMP type. This contains raw UMP data, which can have MIDI 1.0 or
 * MIDI 2.0 packets. The data is an array of 32 bit ints. */
#define JACK_DEFAULT_UMP_TYPE "32 bit raw UMP"

It currently seems to be wired to support UMP internally but then convert to/from 8-bit MIDI 1.0 and not actually exposing the port type to JACK applications (?). Would make sense for the JACK API (and jack2 specifically) to adopt this type as well.

[falkTX]

hmm it would be nice for pipewire to not add its custom extensions on the jack protocol without talking to jack devs first...
this feels a bit like EEE in a way.

with this we soon might have some "jack clients" that are not compatible with original/official jack and only with pipewire.

[jcelerier]

this issue has been open for five years though, it's very reasonable for other devs to look at this thread and conclude that JACK devs do not see MIDI 2 as a priority

[falkTX]

this issue has been open for five years though, it's very reasonable for other devs to look at this thread and conclude that JACK devs do not see MIDI 2 as a priority

it is still the original project and not dead by any means, I should do more for it but as always life stuff gets in the way and need to prioritize other things.

the current situation is a bit unfair, jack2 is unpaid volunteer work, eventually I do some for it as I have free time and no other tasks that take more priority.
but pipewire devs do it as their job, they are always going to be faster.

adding MIDI2 to JACK is not a simple topic, I first need to understand it quite well to be able to make a good decision on the path to follow, for example:

• new jack port type needed?
• if new jack port, can midi1 ports connect to midi2 and vice versa?
• what kind of conversions are needed?
• if not going for a new port type, then likely we need some example code for devs
• do existing tools need to be updated to support midi2? which ones?
• do internal data structures need to change to make space for bigger midi buffers by default?
• do the alsa seq and raw midi backends need update?
• what updates are needed for windows and macos backends?
• what can be done for (Free)BSD?

I am yet to use a single MIDI2 capable hardware or software, making it hard to get actually familiar with the concept. While MIDI2 is not exactly new now, there is a clear lack of capable hw/sw compatible with it.

And also so far there has been only a little bit of discussion around it, I dont recall seeing actual proposals (in terms of code) in regards to MIDI2 support.

[jcelerier]

new jack port type needed?

since pipewire already did it like this I think it's better for everyone to not reinvent and not have to if things depending on the detected runtime jack backend

if new jack port, can midi1 ports connect to midi2 and vice versa?

yes

what kind of conversions are needed?

there's a well defined conversion between MIDI1 and MIDI2 given by the spec ; all operating systems annd multiple libraries implement it. in libremidi I use the one from Atsuhi Eno - my code is very suboptimal but does the job so far, feel free to reuse ! https://github.com/celtera/libremidi/blob/master/include/libremidi/detail/conversion.hpp

if not going for a new port type, then likely we need some example code for devs
do existing tools need to be updated to support midi2? which ones?

they don't need to but for sure people will be starting asking for more and more MIDI 2 features - at least group support, high-resolution CCs etc.

do internal data structures need to change to make space for bigger midi buffers by default?

MIDI 2 is much better than MIDI in that regard - unlike MIDI 1 where you need to think about which buffer size you want because of unbounded sysex size, MIDI 2 packet sizes are fixed, a MIDI2 UMP is at most 4 * 32 bits.

do the alsa seq and raw midi backends need update?

yes - both provide access to UMP.
Feel free to reuse my code in there: https://github.com/celtera/libremidi/tree/master/include/libremidi/backends

what updates are needed for windows and macos backends?

same as above :) both provide new APIs (new functions in CoreMIDI for macOS and WinMIDI for Win32). The new macOS11 api covers both MIDI 1 and MIDI 2 but restricts to macOS 11.0 and later.

what can be done for (Free)BSD? bit

no idea here, I don't know if any BSD has implemented MIDI 2 yet.

Note that all operating systems implement conversion of MIDI 1 to MIDI 2 for MIDI 1 hardware - if someone in 2025 makes an app that uses the MIDI 2 APIs their app is automatically compatible with MIDI 1 devices on mac, windows and linux

[0EVSG]

I can only comment on the MIDI situation in FreeBSD-land. Depending on hardware there is some MIDI (1.0) device functionality and it's possible to connect JACK through an ALSA bridge. But it's all very basic. Don't hold yourself up with that, just go on with Linux / ALSA and we'll find a way.

[falkTX]

I spoke with Wym (from pipewire) today and we agreed on the approach to follow, which is a set of new port flags to indicate if a midi port is midi2 capable.
See 6dd022f

No implementation yet, but I think this is the correct approach going forward. JACK will need to handle some automatic conversions of midi2 -> midi1 port connections, but everything can be made seamless without needing a new port type.

[ReinholdH]

Very interesting discussion. I see MIDI 2.0 coming.
But please do NOT forget Windows. JACK is increasingly used on Windows, too.
Microsoft recently has announced support of MIDI 2.0 on Windows with WinMM. The changes are not trivial compared to the legacy MM interface but worth for JACK to further being used on Windows.

[bbouchez]

Microsoft recently has announced support of MIDI 2.0 on Windows with WinMM. The changes are not trivial compared to the legacy MM interface but worth for JACK to further being used on Windows.

Take care of one thing (I am in contact with the person at Microsoft in charge of the development) : the new MIDI API for Windows will not be based on WinMM. It will run as a service, and this service can talk with WinMM only for MIDI 1.0. For MIDI 2.0 messages (and basically every UMP message), the new interface will need to be used.

And just clarify, as many people makes the confusion : MIDI 2.0 is only a part of UMP. Technically speaking, UMP (Universal MIDI Protocol) is the correct name. UMP packets can contain MIDI 1.0 messages, MIDI 2.0 messages, timestamping messages, etc... And as @jcelerier said, the cool thing is that all these messages are coded as chunks of 1, 2, 3 or 4 uint32_t integers.

[ReinholdH]

Thanks for the clarification. Sorry if I have created a confusion due to my lack of understanding yet how Microsoft does it.

[symdeb]

In case of any development suggest to start from kernel 6.14 as it added some new flags and functionality for raw ports. raw and seq MIDI 2.0 API are very different implementations specially for enumeration and also consider the sequencer API version by this patch https://patchwork.kernel.org/project/alsa-devel/patch/[email protected]/

. While the sequencer is "port" based where each individual port can be opened and closed by the API, for the raw MIDI 2.0 API this is not the case.
See pull request: https://lore.kernel.org/lkml/[email protected]/

Also there may be a few more flag needed for a MIDI 2.0 port capabilities based the MIDI UMP capability:

1. Groupless message support (i.e the sequencer has a dedicated Groupless port, raw does not). even though it possible to send groupless messages to any port right now AFAIK.
2. Static or non-static. The client may use this to understand to receive/send EP/FB notifications.
   https://patchwork.kernel.org/project/alsa-devel/patch/[email protected]/
3. If the port is a MIDI 2.0 legacy speed port or not (m1,0 bits in GTB or from the Endpoint notifications message)
4. For the raw API, if the port is "active" as the raw API will always return all 16 groups per endpoint. (from kernel 6.14) while the actual device may not support all the groups
5. The group number of the port, so that the client knows what the group number must be in the message for the port
6. Rx/Tx Jitter compensation support
7. Number of Sysex8 streams P.S: There is a new ALSA API for UMP Raw virtual port creation .

Port capabilities may change for and endpoint with non-static function blocks, the number of groups as well.
Jack may need to read those messages to update the capabilities.

For the actual MIDI messages, it may just need for jack to add a bit in the event if it's MIDI 2.0 or legacy event. Based on the flag client can determine the packet size based on the UMP packet type in the first 32 bit word and thus how many 32 bit words to read (this is the way it is done in the Windows MIDI 2.0 callback event which is quite similar to the current Jack API). Based on the packet type bit ,Jack can use the MIDI ALSA legacy or UMP API to send data and return to the client if the data was read via UMP or legacy API by this bit.

I am not very knowledgable with the ALSA kernel, but this may be things to consider. it may help (or not) :-) MIDI 2.0 clients can send data to an MIDI 1.0 port but ofcourse a lot of functionality (MIDI 2.0 UMP, jitter compensation, , the new RPN and sysex8 etc.. messages all don't exists in MIDI 1.0)

As for " MIDI 2.0 is only a part of UMP" I guess it should be "UMP is only a part of MIDI 2.0 and MIDI 2.0 also is about the new MIDI-CI specifications, profiles and network MIDI. BTW some people think you can timestamp messages as in a MIDI file, this is not the case, you can send jitter compensation messages to that a receiver knows message timing need to be offset.

Regarding hardware. Amenote uses an Pico on the Protozoa which firmware just runs fine on a standalone Pico off the shelf.

[bbouchez]

As for " MIDI 2.0 is only a part of UMP" I guess it should be "UMP is only a part of MIDI 2.0 and MIDI 2.0 also is about the new MIDI-CI specifications, profiles and network MIDI.

No, UMP means Universal MIDI Protocol. It contains both MIDI 1.0 and MIDI 2.0. Here is how the UMP specification presents it :

The trick with UMP is that all packets (both MIDI 1.0 and MIDI 2.0) are encoded on a 32-bit basis (with 1, 2, 3 or 4 words in a UMP packet). Most MIDI 1.0 are encoded in a single 32-bit packet (with MT field set to 1 or 2, MT=1 being System Common / Realtime MIDI 1.0 message, and MT=2 being a Channel MIDI 1.0 message)
MIDI 1.0 SYSEX are packetized in chunks of MT=3 messages (64 bits UMP message)

MIDI CI and MIDI Profile are upper level protocols, they rely on UMP/SYSEX in order to identify capabilities of connected devices. Typically they run at application level, not the transport (you can perfectly exchange data over Network UMP without dealing with MIDI-CI and Profiles)

[symdeb]

https://midi.org/details-about-midi-2-0-midi-ci-profiles-and-property-exchange-updated-june-2023
Actually UMP is already at 1.1.1.