Mixing and encoding multiple audio streams at the same time

[andreamisu]

I'm looking inside how to master miniaudio library and I'm quite impressed by both implementation and documentation in the header file.

Nonetheless, I have this use case in which I should export and encode a single .wav audio file starting from two input audio sources (2 different devices). I started looking inside the node example for getting my head around it, but I've no file to load in memory and initialize the ma_decoder object hence the audio captured is coming from the device itself.

Might someone point me out at some method for achieving this or else?

Thank you!

[orcmid]

The TL;DR: I guess it is a question of what your goal is. If you want to DIY for the joy of making audio software (say for doing podcasts or live DJ mixing), have fun. Enjoy the journey.

Questions:
You are talking about two mono sources? Or dual stereo sources? And you are talking about live capture?

This sounds like the sort of thing that is handled by audio docks (such as something like Komplete Audio 6 or one of many equivalents) and some free or bundled Digital Audio Workstation software, such as Ableton Live Lite. This depends on what computer platform you are targeting of course, as well as how you are sourcing the audio.

[andreamisu]

@orcmid My goal is to be able to obtain a single .wav file that live captures and merge both microphone and system sound using miniaudio.

While obtaining system audio and microphone (using Blackhole or Wasapi) into two different .wav files using two different devices is clear, it is not when it's the case of merging two different live audio sources, as the header files in the node examples, just manage already present on disk audio files.

Both microphone and system audio can be mono or stereo.

[K0IN]

have you figured it out? I'm also interested in this :)

[mackron]

There's no built-in support for your specific scenario, but you can certainly do it. Create your two capture devices, output them to their own buffers, when both buffers go beyond a certain threshold, mix them and write them out to the wav file. The annoying part with all of that is the synchronization of each of the devices. They'll both be running on their own threads so you'll need to account for subtle timing differences. Also, starting and stopping won't be atomic so you might need to think of a way to mitigate timing differences in your start/stop times (maybe start and stop your devices, but only start writing data when a global variable is flagged or something).

[mackron]

That's easy - configure your devices so they have the same format/channels/rate:

ma_device_config config1 = ma_device_config_init(...);
config1.capture.format = outputFormat;
config1.capture.channels  outputChannels;
config1.sampleRate = outputSampleRate;

ma_device_config config2 = ma_device_config_init(...);
config2.capture.format = outputFormat;
config2.capture.channels  outputChannels;
config2.sampleRate = outputSampleRate;

[andreamisu]

And what if two devices don't support the same channel / rate ? 👀👀

[mackron]

miniaudio does automatic conversion. It will try using the format you request, but if it's not supported natively, it'll automatically convert it. The data you receive in the data callback is always in the format you request.

[andreamisu]

Hey guys! thank you for the input! It's pretty much everything done and I can also release a small example for the repo after cleaning it, but I have a further question:

• I'm trying to mix together these two ma_format_f32, 2, 44100 sources together, and I saw that in certain cases, just mixing together frame by frame using "+", the ma_uint32 overflows, and the file is then corrupted.

Do we have a more reliable mixing tool besides adding those 2 ma_uint32 sources? or what should be the best practice for that?

[mackron]

You mentioned ma_format_f32, so where does ma_uint32 come into it? miniaudio does not support unsigned 32-bit samples. Do you mean you're mixing ma_format_s32? In any case, no there's no helper APIs for mixing anything other than f32. It's too simple and niche to justify maintaining a public API for that. To mix a 32-bit integer format, just sum into a 64-bit variable (make sure you cast your samples to 64-bit first so the compiler does a 64-bit sum), then clamp it to the minimum and maximum 32-bit value, then cast it back to 32-bit.

If you mean an example for the miniaudio repo, I'm not interested in adding an example for this particular one I'm sorry.