Use this to create a large ML corpus of synth sounds and their parameters?

[turian]

Is there a simple way to create a large ML corpus (using Python) to generate a large corpus of synth sounds and their parameters? I am trying to learn synth parameters from a given audio file.

[jcelerier]

hi ! I'm sorry, I'm really not an ML expert. this library can be used to easily generate bindings to either existing or new algorithms written in C++, into other languages. If you are doing AI / ML in python the algorithms already exist in e.g. tensorflow, pytorch, etc as far as I know. Paging @ktatar who is an AI expert and will certainly know much better than me :)

[turian]

@jcelerier @ktatar Hey! So I'm not so worried about the ML part.

What I want is a reasonably powerful software synth, where the parameters are in [0, 1] (hopefully in a sensible range, and then just generate a huge dataset of:

(audio, parameters used to generate that audio)

where there are a fixed number of parameters.

[ktatar]

Hey @turian, there is quite a bit of different ways to do that. There is a recent one running on gpu, and I suspect that this one may be suitable for what you would like to do:

https://github.com/torchsynth/torchsynth

Happy to chat on another channel if you would like to discuss further: https://kivanctatar.com/contact

[ktatar]

@turian I just saw that you already published a cool paper building on torchsynth, so I guess I was too late to answer your question. The paper looks cool! I will check it out.

[ktatar]

oh wait, original torchsynth paper is your work too @turian ! I am slightly confused, maybe I got your question wrong. Doesn't torchsynth generate sound-parameter pairs already?

[turian]

@ktatar @jcelerier There are several reasons we are interested in non-torchsynth ways of creating synth corpora:

1. Transfer learning to new synths and low-shot learning
2. Synth modules and possible sound design that are not yet part of torchsynth

[ktatar]

There is some open-source c++ synths that are based on known designs such as DX-7. Would that be sth that interests you?

I noticed a python library where you could also import a VST to python, but I haven't tried the feasibility of this yet.

Lastly, I would say the biggest caveat of torchsynth is that it is not made for realtime synthesis, which is a bummer.

I am happy to have a zoom chat, maybe that is easier?

[turian]

@KTatER Happy to zoom yes. Email is lastname at google's email service.

We've using a DX7 emulator in our torchsynth paper and released the dataset. Note that DX7 doesn't have any noise.

I've persuaded the authors of Surge synth to add python binding.

The DAW hosts you might be talking about are:

• Spiegelib written by co-author @jorshi which wraps the (non-maintained) RenderMan library by @fedden. I seem to recall it only supports Python 2.
• MrsWatson by @terragonaudio which I've used with varying luck in the past. Discontinued.
• DawDreamer, a promising new project which has improving compability by @DBraun