Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Audio (Table provided mechanical audio) #57

Open
2 of 3 tasks
freezy opened this issue Sep 11, 2019 · 7 comments
Open
2 of 3 tasks

Implement Audio (Table provided mechanical audio) #57

freezy opened this issue Sep 11, 2019 · 7 comments
Labels
audio Audio-related issue
Milestone

Comments

@freezy
Copy link
Member

freezy commented Sep 11, 2019

Audio is currently completely absent, even on parse level, i.e. the audio streams are skipped. What needs to be done:

  • Parse the audio data from the VPX file
  • Make them accessible by name in the table object
  • Add a JavaScript API to play them, with the parameters Visual Pinball provides, namely:
    • loop count
    • volume
    • pan,
    • random pitch
    • pitch,
    • use same (?)
    • restart
    • front rear fade

We might be able to stream the audio directly from the table blob. To check. A good high-level audio library seems to be Howler.

@freezy freezy added the audio Audio-related issue label Sep 11, 2019
@freezy freezy added this to the First Milestone milestone Sep 12, 2019
@neophob
Copy link
Contributor

neophob commented Nov 4, 2019

How does VPX implements audio? is the ROM used for feedback?

@freezy
Copy link
Member Author

freezy commented Nov 4, 2019

A .vpx file ships with a bunch of .wav or MP3 files, and the table script plays them. That's for mechanical sound effects, the actual ROM sound isn't handled at all, that's done by PinMAME.

@neophob
Copy link
Contributor

neophob commented Nov 4, 2019

Ok so this is about handling the mechanical sound effects then, right?

@neophob neophob changed the title Implement Audio Implement Audio (Table provided mechanical audio) Nov 4, 2019
@freezy
Copy link
Member Author

freezy commented Nov 4, 2019

This issue, yes, entirely.

@freezy
Copy link
Member Author

freezy commented Nov 24, 2019

I've added the parsing to feature/sound. What's left to do is implement the scripting API:

TODO - Reference

There's another way of playing sounds however, which is through external files in VP's Music folder. I think that's what PlayMusic uses, but I'll verify. In this case we would need a special provider whose implementation changes depending on the platform or environment.

@freezy
Copy link
Member Author

freezy commented Nov 24, 2019

Note that the table loader doesn't actually load the sound yet - it just parses its metadata and saves the position where the sound data is stored within the stream. All the data is in PinSoundData, while access goes through PinSound, which can be found in the Table's sounds property.

It might be a good idea to link the data via a Blob to an URL with createObjectURL, so it's accessible to any sound library. Like it's currently done with textures as well.

@freezy
Copy link
Member Author

freezy commented Nov 25, 2019

About pre-loading, here's how it currently works with textures:

  • The public VPX-JS API to create the 3D scene is generateTableNode(). It's currently called like that because it creates a node-base scene and I didn't want to use three.js lingo (better suggestions welcome).
  • I would plug into the same method. It's basically what vpweb calls when the table is loaded. You could also create another method, but then you somehow got to make sure the host app doesn't forget to call it.
  • For textures I'm using two different loaders, ThreeTextureLoaderNode and ThreeTextureLoaderBrowser. This is because Node.js does additionally optimize and convert the images and uses the fs API, while the browser does no optimizations and uses an URL loader. You'll probably have to do the same, because createObjectURL() doesn't exist in Node.js (just stub it with nothing in Node.js).

So your pre-loading would loop through the table's sound properties, potentially pre-process them with volume and whatnot (check what VP does), create object URLs to the data, and save the link to it in the PinSound object.

Your Howler implementation can then work with those URLs as if they were normal remote URLs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
audio Audio-related issue
Projects
None yet
Development

No branches or pull requests

2 participants