This is an Android experiment submission
Firework were invented in ancient China in the 12th century. When I was wondering how to generate an amazing firework show on the mobile device, canvas-fireworks showcases the range of possibilities with HTML5 canvas. Why not creating an experiement with Android?
Besides seeing the world through our eyes, we sense the world via sound as well. In this experiment, I would like to demo how a simple audio processing techniques can make the phone more useful.
The app utilizes the microphone and analyses the real-time audio input. It will build up an audio profile about the environment and use the profile to identify the boost of audio power (volume). Whenever the app detects the audio level beyond expected range, it will fire a firework into the sky.
It recommends to start the app in a relatively quite environment and use earphone as audio output.
The key to simulate fireworks is to establish the 3D space behind the explosion. While simulating the physics in 3D, I map the 3D space into 2D canvas with corresponding normalized gravity, air resistance and wind effect. Most of physical explaination can be found from Firework Simulation by Bell el.
The app collects the audio in the form of normalized values and perform a simple analysis called 'Normalized Power Sequences'. It works in five steps
- Estimation of the signal power, for each consecutive non-overlapping block of audio sample
- Windowing of the obtained power sequence to consider only its more recent elements
- Normalization of the windowed power sequence
- Determination of the variance of the resulting normalizing sequence
- Application of a threshold on the variance for making the detection decsion
More information can be found via Detection and Recognition of Impulsive Sound Signals by Alain Dufaux. It is possible to apply more techniques discussed in the paper, i.e. use FFT to identify different sound pattern, but the complexity is beyond this experiment. I include the FFT implementation as a sub module as it is quite handy to use. More example about the FFT analysis can be found via this repo.
The audio sample comes from freesound.org, a Creative Commons Licensed audio database.
Since this is an Android experiment, I aim to achieve the reality via "Android approach", without any third party graphic library or openGL knowledge.
- support Android 4.4+
- audio capture via microphone
- surface & canvas drawing
appan Android Studio projectspectrum-analyseinclude a FFT implementation for feature analysis