diff --git a/docs/_sidebar.md b/docs/_sidebar.md index 16720858..a38aabc4 100644 --- a/docs/_sidebar.md +++ b/docs/_sidebar.md @@ -28,3 +28,4 @@ - [Language](/plugins/language.md) - [Theme](/plugins/theme.md) - [Global App State](/plugins/global_app_state.md) + - [Audio](/plugins/audio.md) diff --git a/docs/plugins/audio.md b/docs/plugins/audio.md new file mode 100644 index 00000000..4fd03276 --- /dev/null +++ b/docs/plugins/audio.md @@ -0,0 +1,168 @@ + +# Audio Plugin + +The Blits Audio Plugin allows developers to integrate audio playback into their Blits applications. This plugin provides a simple API for preloading, playing, controlling, and managing audio tracks, including managing volume, playback rate (pitch), and other settings. + +**Note:** When testing or developing on Chrome, audio may not start immediately due to browser restrictions on `AudioContext`. You might see the following error: +`The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu`. This issue occurs on desktop during development but is **not** an issue on Smart TVs, STBs, or Game Consoles. Once you interact with the application (e.g., click or press a key), the error will go away, and sound playback will function properly. + +## Registering the Plugin + +The Audio Plugin is not included by default and needs to be explicitly registered before usage. This makes the plugin _tree-shakable_, meaning if audio is not required, it won't be part of the final app bundle. + +To register the plugin, you should import and register it before calling the `Blits.Launch()` method, as shown in the example below: + +```js +// index.js + +import Blits from '@lightningjs/blits' +// import the audio plugin +import { audio } from '@lightningjs/blits/plugins' + +import App from './App.js' + +// Register the audio plugin with optional preload settings +Blits.Plugin(audio, { + preload: { + background: '/assets/audio/background.mp3', + jump: '/assets/audio/jump.mp3', + }, +}) + +Blits.Launch(App, 'app', { + // launch settings +}) +``` + +The Audio Plugin can accept an optional `preload` configuration, which allows you to preload audio files during initialization. These files are stored in an internal library for easy access during gameplay. + +## Playing Audio Tracks + +Once the plugin is registered, you can play audio tracks either from the preloaded library or from a URL. Here’s an example of how to use it inside a Blits Component: + +```js +Blits.Component('MyComponent', { + hooks: { + ready() { + // Play a preloaded track and get a track controller + const bgMusic = this.$audio.playTrack('background', { volume: 0.5 }, 'bg-music') + + // Play a track from URL and get its controller + const effect = this.$audio.playUrl('/assets/audio/victory.mp3', { volume: 0.8 }) + }, + }, +}) +``` + +The `playTrack()` method allows you to play an audio track from the preloaded library, while `playUrl()` allows you to play a track from a specified URL. Both methods return a track controller object. + +### Track Controller Methods: +- `pause()`: Pauses the track. +- `resume()`: Resumes the paused track. +- `stop()`: Stops the track and removes it from the active list. +- `setVolume(volume)`: Adjusts the playback volume for the track. + +### Example Usage of Track Controller: +```js +Blits.Component('MyComponent', { + hooks: { + ready() { + const bgMusic = this.$audio.playTrack('background', { volume: 0.5 }, 'bg-music') + + // Pause, resume, and set volume on the track + bgMusic.pause() + bgMusic.resume() + bgMusic.setVolume(0.8) + bgMusic.stop() + }, + }, +}) +``` + +## Preloading Audio Files + +The most efficient way to manage audio in your app is to preload audio files. The Audio Plugin supports preloading via the `preloadTracks()` method. You can pass in an object where each key is the track name, and each value is the URL of the audio file. + +```js +Blits.Component('MyComponent', { + hooks: { + init() { + this.$audio.preload({ + jump: '/assets/audio/jump.mp3', + hit: '/assets/audio/hit.mp3', + }) + }, + }, +}) +``` + +Preloaded audio files are stored in an internal library, which you can reference when calling `playTrack()`. + +## Removing Preloaded Audio Tracks + +In some cases, you might want to remove a preloaded audio track from the library, freeing up memory or resources. You can do this using the `removeTrack()` method: + +```js +Blits.Component('MyComponent', { + input: { + removeJumpTrack() { + // Remove the 'jump' track from the preloaded library + this.$audio.removeTrack('jump') + }, + }, +}) +``` + +The `removeTrack(key)` method deletes the specified track from the internal `tracks` object, preventing further access to it. + +## Error Handling + +In cases where the `AudioContext` cannot be instantiated (e.g., due to browser limitations or disabled audio features), the Audio Plugin will automatically disable itself, preventing errors. If the `AudioContext` fails to initialize, an error message will be logged, and audio-related methods will return early without throwing additional errors. + +You can check whether audio is available via the `audioEnabled` property: + +```js +Blits.Component('MyComponent', { + hooks: { + ready() { + if (!this.$audio.audioEnabled) { + console.warn('Audio is disabled on this platform.') + } + }, + }, +}) +``` + +This ensures that your app continues to function even if audio features are not supported or available. + +## Public API + +The Audio Plugin provides the following methods and properties: + +- `playTrack(key, { volume, pitch }, trackId)`: Plays a preloaded audio track and returns a track controller. +- `playUrl(url, { volume, pitch }, trackId)`: Plays an audio track from a URL and returns a track controller. +- `pause()`: Pauses the current audio track. +- `resume()`: Resumes the current audio track. +- `stop(trackId)`: Stops a specific audio track by its ID. +- `stopAll()`: Stops all currently playing audio tracks. +- `setVolume(trackId, volume)`: Sets the volume for a specific track by its ID. +- `preload(tracks)`: Preloads a set of audio tracks into the internal library. +- `removeTrack(key)`: Removes a preloaded track from the library. +- `destroy()`: Destroys the audio context and stops all tracks. +- `getActiveTracks`: Return a list of active track IDs +- `getActiveTrackById(trackId)`: Get an active track by its ID, returns `null` if not found (or stopped). +- `get audioEnabled`: Returns `true` if the `AudioContext` is available and audio is enabled. + +## Destroying the Plugin + +When you're done with the audio functionality, you can clean up the plugin and close the `AudioContext` by calling the `destroy()` method. This is especially useful when you no longer need audio in your application: + +```js +Blits.Component('MyComponent', { + hooks: { + exit() { + this.$audio.destroy() + }, + }, +}) +``` diff --git a/src/plugins/audio.js b/src/plugins/audio.js new file mode 100644 index 00000000..2302fc9b --- /dev/null +++ b/src/plugins/audio.js @@ -0,0 +1,239 @@ +/* + * Copyright 2024 Comcast Cable Communications Management, LLC + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ + +import { Log } from '../lib/log.js' + +export default { + name: 'audio', + plugin(options = {}) { + let audioContext = undefined + let audioEnabled = true + const activeTracks = {} // Store all active track controllers + const tracks = {} + + const init = () => { + if (audioEnabled === false) return + + try { + audioContext = new AudioContext() + const testSource = audioContext.createBufferSource() + audioEnabled = true + } catch (e) { + Log.error('AudioContext is not supported or failed to initialize. Audio will be disabled.') + audioEnabled = false + + // Attempt to re-initialize on a user gesture (e.g., a click) + window.onclick = () => { + init() + } + } + + // Preload tracks if options.preload is provided + if (audioEnabled && options.preload && typeof options.preload === 'object') { + preloadTracks(options.preload) + } + } + + const loadAudioData = async (url) => { + if (audioEnabled === false) return + try { + const response = await fetch(url) + + if (!response.ok) { + throw Error(`${response.status} - ${response.statusText}`) + } + + const arrayBuffer = await response.arrayBuffer() + return audioContext.decodeAudioData(arrayBuffer) + } catch (e) { + Log.error(`Failed to load audio from ${url}: ${e}`) + } + } + + const preloadTracks = async (trackList) => { + Log.info('Preloading tracks...') + for (const [key, url] of Object.entries(trackList)) { + const audioData = await loadAudioData(url) + if (audioData) { + tracks[key] = audioData + } + } + Log.info('Preloading completed.') + } + + const createTrackController = (source, gainNode, trackId, options = {}) => { + const trackController = { + stop() { + try { + source.stop() + } catch (e) { + Log.warn('Error stopping audio track', trackId) + } + + delete activeTracks[trackId] + }, + setVolume(volume) { + gainNode.gain.value = volume + }, + get source() { + return source + }, + get gainNode() { + return gainNode + }, + } + + // Handle loop option + if (options.loop === true) { + source.loop = true + } + + // Always remove from activeTracks on 'ended', then call the provided callback (if any) + source.onended = () => { + delete activeTracks[trackId] + if (typeof options.onEnded === 'function') { + options.onEnded() + } + } + + return trackController + } + + const playAudioBuffer = ( + buffer, + trackId, + { volume = 1, pitch = 1, loop = false, onEnded = null } = {} + ) => { + if (audioEnabled === false || audioContext === undefined) { + Log.warn('AudioContext not available. Cannot play audio.') + return + } + + const source = audioContext.createBufferSource() + source.buffer = buffer + source.playbackRate.value = pitch + + const gainNode = audioContext.createGain() + gainNode.gain.value = volume + + source.connect(gainNode) + gainNode.connect(audioContext.destination) + + // Create and store the track controller + const trackController = createTrackController(source, gainNode, trackId, { loop, onEnded }) + activeTracks[trackId] = trackController + + source.start() + + return trackController + } + + const playTrack = (key, options = {}, trackId = key) => { + if (audioEnabled === false) { + Log.warn('AudioContext not available. Cannot play track.') + return + } + + if (tracks[key] !== undefined) { + return playAudioBuffer(tracks[key], trackId, options) + } + + Log.warn(`Track ${key} not found in the library.`) + return null + } + + const playUrl = async (url, options = {}, trackId = url) => { + if (audioEnabled === false) return + const audioData = await loadAudioData(url) + if (audioData !== undefined) { + return playAudioBuffer(audioData, trackId, options) + } + } + + const stop = (trackId) => { + if (audioEnabled === false || activeTracks[trackId] === undefined) return + activeTracks[trackId].stop() + } + + const stopAll = () => { + if (audioEnabled === false) return + while (Object.keys(activeTracks).length > 0) { + const trackId = Object.keys(activeTracks)[0] + stop(trackId) + } + } + + const removeTrack = (key) => { + if (tracks[key] !== undefined) { + // stop if the track happens to be active as well + if (activeTracks[key] !== undefined) { + activeTracks[key].stop() + } + + delete tracks[key] + Log.info(`Track ${key} removed from the preloaded library.`) + } else { + Log.warn(`Track ${key} not found in the library.`) + } + } + + const getActiveTracks = () => { + return Object.keys(activeTracks) + } + + const getActiveTrackById = (trackId) => { + return activeTracks[trackId] || null + } + + const destroy = () => { + if (audioEnabled === false) return + stopAll() // Stop all active tracks before destroying + audioContext.close() + } + + // Attempt initialization and preload + init() + + // Public API for the Audio Plugin + return { + get audioEnabled() { + return audioEnabled + }, + get tracks() { + return tracks + }, + get state() { + return audioContext.state + }, + getActiveTracks, // Return a list of active track IDs + getActiveTrackById, // Return active track by its ID or null + destroy, // Destroy the audio context and stop all tracks + pause() { + return audioContext.suspend() + }, + playTrack, // Play a preloaded track by its key and return the track controller + playUrl, // Play a track directly from a URL and return the track controller + preload: preloadTracks, // Preload a set of audio tracks + resume() { + return audioContext.resume() + }, + removeTrack, // Remove a track from the preloaded library + stop, // Stop a specific track by its ID + stopAll, // Stop all active tracks + } + }, +} diff --git a/src/plugins/audio.test.js b/src/plugins/audio.test.js new file mode 100644 index 00000000..166997af --- /dev/null +++ b/src/plugins/audio.test.js @@ -0,0 +1,206 @@ +/* + * Copyright 2024 Comcast Cable Communications Management, LLC + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ + +import test from 'tape' +import audio from './audio.js' +import { initLog } from '../lib/log.js' +import Settings from '../settings.js' + +// Enable debug logging +Settings.set('debugLevel', 4) +initLog() + +// Mock AudioContext and its methods +class MockAudioContext { + constructor() { + this.state = 'suspended' + } + + resume() { + this.state = 'running' + } + + suspend() { + this.state = 'suspended' + } + + decodeAudioData(buffer) { + return buffer + } + + createBufferSource() { + return { + connect: () => {}, + start: () => {}, + stop: () => {}, + playbackRate: { value: 1 }, + onended: null, + } + } + + createGain() { + return { + gain: { value: 1 }, + connect: () => {}, + } + } + + close() { + return Promise.resolve() + } +} + +// Mock some globals +global.window = { + console, +} +global.AudioContext = MockAudioContext +global.fetch = () => + Promise.resolve({ + ok: true, + arrayBuffer: () => Promise.resolve(new ArrayBuffer(8)), + }) + +test('Audio Plugin - Initialization', (assert) => { + const plugin = audio.plugin() + assert.equal(plugin.audioEnabled, true, 'Audio should be enabled if AudioContext is available') + assert.end() +}) + +test('Audio Plugin - Preload tracks', async (assert) => { + const plugin = audio.plugin() + await plugin.preload({ + track1: '/audio/track1.wav', + track2: '/audio/track2.wav', + }) + assert.pass('Tracks should preload without errors') + assert.end() +}) + +test('Audio Plugin - Play a preloaded track', async (assert) => { + const plugin = audio.plugin() + + await plugin.preload({ + track1: '/audio/track1.wav', + }) + + const track = plugin.playTrack('track1', { volume: 0.5 }, 'track1') + assert.equal(plugin.getActiveTrackById('track1') !== null, true, 'Active track should exist') + + assert.ok(track.stop, 'Track controller should have stop method') + assert.end() +}) + +test('Audio Plugin - Play a track from URL', async (assert) => { + const plugin = audio.plugin() + + const track = await plugin.playUrl('/audio/test.wav', { volume: 0.8 }) + assert.equal( + plugin.getActiveTrackById('/audio/test.wav') !== null, + true, + 'Active track should exist' + ) + + assert.ok(track.stop, 'Track controller should have stop method') + assert.end() +}) + +test('Audio Plugin - Pause, Resume, and Stop', async (assert) => { + const plugin = audio.plugin() + + await plugin.preload({ + track1: '/audio/track1.wav', + }) + + const track = plugin.playTrack('track1', { volume: 0.5 }, 'track1') + assert.equal(plugin.getActiveTrackById('track1') !== null, true, 'Active track should exist') + + // Pause + plugin.pause() + assert.equal(plugin.state === 'suspended', true, 'Track should pause successfully') + + // Resume + plugin.resume() + assert.equal(plugin.state === 'running', true, 'Track should resume successfully') + + // Stop + track.stop() + assert.equal( + plugin.getActiveTrackById('track1'), + null, + 'Track should be removed from active tracks after stopping' + ) + assert.pass('Track should stop successfully') + assert.end() +}) + +test('Audio Plugin - Stop all tracks', async (assert) => { + const plugin = audio.plugin() + + await plugin.preload({ + track1: '/audio/track1.wav', + track2: '/audio/track2.wav', + }) + + plugin.playTrack('track1', { volume: 0.5 }, 'track1') + plugin.playTrack('track2', { volume: 0.5 }, 'track2') + + assert.equal( + plugin.getActiveTrackById('track1') !== null && plugin.getActiveTrackById('track2') !== null, + true, + 'Both tracks should be playing' + ) + + plugin.stopAll() + + assert.equal( + plugin.getActiveTrackById('track1') === null && plugin.getActiveTrackById('track2') === null, + true, + 'Both tracks should be stopped' + ) + assert.pass('All tracks should stop successfully') + assert.end() +}) + +test('Audio Plugin - Remove a preloaded track', async (assert) => { + const plugin = audio.plugin() + + await plugin.preload({ + track1: '/audio/track1.wav', + }) + + plugin.removeTrack('track1') + + const preloadedTracks = plugin.tracks + + assert.equal(preloadedTracks.track1, undefined, 'Track 1 should be removed from preloaded Tracks') + assert.equal(plugin.playTrack('track1'), null, 'Preloaded track should be removed') + assert.end() +}) + +test('Audio Plugin - Destroy the plugin', async (assert) => { + const plugin = audio.plugin() + + await plugin.preload({ + track1: '/audio/track1.wav', + }) + + plugin.destroy() + + assert.pass('Plugin should destroy and stop all tracks') + assert.end() +}) diff --git a/src/plugins/index.js b/src/plugins/index.js index 851dba7a..c025d4ad 100644 --- a/src/plugins/index.js +++ b/src/plugins/index.js @@ -18,3 +18,4 @@ export { default as language } from './language.js' export { default as theme } from './theme.js' export { default as appState } from './appstate.js' +export { default as audio } from './audio.js'