Automatically generating a music video by extracting scenes from another video.
You can try it without installing anything by using this colab.
- Python 3.6 or newer
- A few python libraries (Try something like
pip install -r requirements.txt)
If the video contains music (i.e it's a music video) the program is going to take advantage of that identifying sections like verse, chorus and bridges, those sections are saved into temp folder, then scendetect splits these into short clips and saves the result into the scenes folder.
The script has 3 parameters. VIDEO_PATH, is the video we want to process. SCENE_SPLITTER_SENSITIVITY is an optional integer value, it's for tuning the sensitivity of the scene splitter, it's 20 by default. IS_MUSIC_VIDEO is an optional boolean value, it's True by default.
python scenes_preprocessing.py VIDEO_PATH SCENE_SPLITTER_SENSITIVITY IS_MUSIC_VIDEO
python scenes_preprocessing.py samples/MidnightCity.mp4
This will generate folders containing the clips inside the scenes folder.
Before generating the final video it's a good idea to check if the generated clips look fine. You can delete any clip that you don't like. You can also just mix or delete scenes folders.
Once we have our folder with video scenes we use it to create a music video. The program first idenfifies the song sections, and calls them "A", "B", "C", ... which map to the folders that contain the scenes. Then it identifies the rhythm of the song and changes of scene on every bar.
python music_video.py PATH_TOFOLDER_WHERE_SCENES_ARE PATH_TO_SONG
python music_video.py scenes samples/TimeToPretend.mp3
This will generate a video called output
This source code was developed by Mathias Gatti (@mathigatti) if you use it please remember to cite me.
Mathias's open-source projects are supported by his Patreon. If you found this project helpful, any monetary contributions to the Patreon are appreciated and will be put to good creative use.
MIT