Brainharmonic - Generating music from brain signal data using deep learning

[arianguyen]

Title

Brainharmonic - Generating music from brain signal data using deep learning

Leaders

• Mariano Cabezas
• Aria Nguyen

Collaborators

No response

Brainhack Global 2021 Event

Brainhack Australasia

Project Description

This project aims to develop a tool to generate music from brain signals using deep learning models. There has been a lot of work on generating new music from a large collection of music using deep AI models, as well as some work on generating music from brain EEG/fMRI signal through algorithmic or rule based approaches. However there has not been work done on using deep learning models to generate music from EEG/fMRI signal.

In this project, we will use deep generative models to allow EEG/fMRI data to be used as an input to generate music. We will also look into linking brain signal from different regions of the brain to different instruments to create a symphony. More details of the design can be found on our Github page.

Link to project repository/sources

https://github.com/marianocabezas/brainharmonic

Goals for Brainhack Global

Develop a tool to generate songs from EGG/fMRI brain signals using deep learning

Good first issues

1. Issue one:
   Develop a generative deep learning model to generate songs from a latent space
2. Issue two:
   Develop a deep learning model to translate EEG/fMRI sequences from similar subjects into a latent space

Communication channels

https://mattermost.brainhack.org/brainhack/channels/brainharmonic

Skills

• Python: intermediate
• Pytorch: beginner
• EEG/fMRI data processing: beginner

Onboarding documentation

No response

What will participants learn?

• EEG/fRMI data preprocessing
• Deep learning generative models
• Clustering/classification modelling techniques
• Existing music-to-music deep AI frameworks

Data to use

No response

Number of collaborators

2

Credit to collaborators

Project contributors are credited on Readme file on project Github page

Image

Type

coding_methods, data_management, method_development, pipeline_development

Development status

0_concept_no_content

Topic

deep_learning, information_theory, machine_learning

Tools

fMRIPrep, Freesurfer

Programming language

Python

Modalities

EEG, fMRI

Git skills

1_commit_push, 2_branches_PRs

Anything else?

No response

Things to do after the project is submitted.

• Twitter-sized summary of your project pitch.

[complexbrains]

Dear @arianguyen thank you very much for submitting this very intriguing project! 🎉 As a simultaneous EEG/fMRI enthusiast, I am very curious to see how the project will evolve over time. 🤗

Everything seems ready, but maybe missing an image for your project if you have one. If you want us to use a specific one please just drop it anywhere in the issue and let us know that your project is ready to go! But we can also publish it with a template image if you are ok with it, but you can always edit your issue after it is published too.

Looking forward to hearing more about this amazing project! 🚀

[arianguyen]

Hi Isil, Thank you for your message. Yes @mariano and I are very excited about this project! We are working on a logo for the project at the moment but otherwise it is ready, if you can help us publish the project with a template image for now that would be great. We will edit the issue later after we finalise the logo. Cheers, Aria

…

On Sun, 28 Nov 2021 at 5:44 am, Isil Bilgin ***@***.***> wrote: Dear @arianguyen <https://github.com/arianguyen> thank you very much for submitting this very intriguing project! 🎉 As a simultaneous EEG/fMRI enthusiast, I am very curious to see how the project will evolve over time. 🤗 Everything seems ready, but maybe missing an image for your project if you have one. If you want us to use a specific one please just drop it anywhere in the issue and let us know that your project is ready to go! But we can also publish it with a template image if you are ok with it, but you can always edit your issue after it is published too. Looking forward to hearing more about this amazing project! 🚀 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#135 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGUAZKWKWNILUQTCC6BBTELUOERKHANCNFSM5IZ3ZK3A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[complexbrains]

Thank you for your reply dear Aria! Indeed! Such a lovely project to propose! Really excited for you both!

Surely I will make it published but you can make the changes in any part of the issue as you wish whenever you want!

Hope you all enjoy your participation in Brainhack! 🤗

[arianguyen]

Hi,

We have created a logo for this project. Could you please help include that on the website? Thank you!

Best,
Aria

[complexbrains]

Oh thats lovely @arianguyen. Let's make it come alive in Brainhack website then! I am pretty excited for you both!

Good luck with your project 🍀