Before the user testing, I decided that having an in depth look at both of the plug-ins. This would allow me to make my own judgments and decided what I would personally go for in terms of the project. This isn’t about what is better, it is about what is more beneficial at creating a realistic sense of presence within the virtual scene
Since the early beta stage, there has been plenty of documentation to refer upon when integrating Steam audio. However, within 2017, there were many inconsistencies when referring to the documentation within Unreal engine. For instance, certain drop-down menu parameters didn’t exist when following the guidelines. This was probably due to it being in its beta stage which has been solved now. Within Unity on the other hand, there were no problems at all. Tagging the geometry and constantly pre-baking the scene can be a nuisance, but the results are worth it. Finally, a big win for steam is that there are no requirements for any type of audio files. Allowing any mono recordings to be utilised with the steam audio source scripts.
One thing that is instantly noticeable is the occlusion parameters within Steam Audio. Linked up with real-time reflections, allows the sound to travel around obstacles and not through them. Occlusion parameters also help define what and where solid objects are allowing the audio to seamlessly travel throughout the environment. “Steam Audio can model how sound is occluded by, and passes through, solid objects” (Valvesoftware.github.io, 2018). Additionally, having frequency dampening controls and even indirect mix level controls allow one to have control of how the sound moves freely from the source. The only downside is that it is quite difficult to determine if height is above or below within the HRTF’s. Even on the highest setting for billinear tracking, it takes a lot of focus and listening to hear the effects. (Below is a steam audio source component attached to the train model.)
The reflections create a realistic feeling in terms of walls and objects occlusion. “Steam Audio has the best sounding occlusion system by far, allowing gradual amounts of occlusion and frequency-dependent transmission through objects.” (Gould, 2018) Having sound travel around objects makes the difference. Reverberation and tagging the geometry helps create a realistic sense of presence however, the lack of accurate head tracking can break the presence if there are few sounds within the scene. Finally, another noticeable flaw is the delay from walking from one room to another. It does take about a second for the transition to happen in terms of the audio shift. Steam audio does seem like an ideal candidate in terms or realistic reflections but, what does the recent Resonance plug-in have to offer that Steam may not?
Google resonance got its first major update early this year. After watching the demo video, it showed its potential right away. The document was easy to follow although, Resonance only accepts ambiosonic files. “Note that the selected Ambisonic file should be in ACN/SN3D format” (Google Developers, 2018) This requirement makes the integration a lot more difficult in terms of the project. Most of the sounds that are needed must be realistic and mechanical, for instance: A large subway train, traffic beeping and helicopter whirring noises etc. This limit’s resonance’s potential. Although, having one environment such as a church and having ambisonic choral singing would be ideal in this type of scenario. Perhaps Resonance will be better suited within 360 virtual experiences rather than virtual games.
The ultimate ability Resonance has is it’s ‘superb’ head tracking feature. As soon as you move the camera within the demo, it is extremely accurate in terms of panning and audio directivity based on your distance and head position. “Resonance Audio uses “highly optimized digital signal processing algorithms” based on Ambisonics that will allow for spatialization of hundreds of sounds at once, without compromising quality” (Deahl, 2018). The reverb sounds incredibly authentic as well, especially when you move from the church to the cave. The transition is almost natural and smooth unlike steam’s version of reverb transitions. However, one problem with the functionality of resonance, is that the reflections and occlusion don’t have much ‘punch.’ The occlusion parameter only dampens the sound rather than allowing for the sound to travel around or past walls. The only available control for the occlusion parameter is the intensity.
There is no doubt that Google’s resonance has the best HRTF’s. The transition from each room is also impresssive. However, the limitations for reflections and occlusion hinder it’s potential. “Occlusion is a pretty basic on/off (with a single settable intensity) which can be unrealistic.” (Gould, 2018) Although the head tracking is unlike any other tool out there at the moment, it still only allows you to input ambiosonic files. Again, this would be suited perfectly for virtual experiences rather than a realistic game scene.
Both have promising features for the project. Having tested them both numerous times, choosing Steam audio is the better option. It matches the project’s requirements perfectly. The sound’s need to be highly detailed and realistic and they also need to reflect and bounce around the city’s walls. The occlusion will also improve the presence for each area- whether it being within a shop or standing outside- they need to sound coherently different. The only downside to not having resonance would be its head tracking. But it is noticeable within steam. This project is about creating an overall sense of presence within a virtual scene and Steam has the best use of tools and integration more so than Resonance. A blend between the two would fix each disadvantage contained within the other.
The decision of the spatialization plug-in had been decided, setting up the scripts and integrating the audio sources now became the primary issue. After having the sounds linked up to the ongoing animation of the train within the scene and having the other sound sources placed around the area, it was time to start integrating Steam audio. Having the previous knowledge of how to setup the scripts allowed the project to be updated instantly. However, as new art assets were added, updating the geometry every time proved to be time consuming. Instead, waiting for the artist to fully tag a generic area such as- ‘food court’ or ‘noodle bar’ allowed for the geometry to be tagged all at once, as well as selecting the appropriate material. Once this was done, setting up the properties within the scripts for each sound source didn’t take long at all. Setting the reverb to real-time as well instead of baking the reverb probes allowed for testing the general reflections of the area. Once the different sources began to build up, testing them within virtual reality was the way forward. This would then make iteration every week easier. Doing multiple tests throughout the week and tweaking the sources enabled improvements to rise when testing it out in the Vive every Friday.
Now that the final preparations for the project had begun, gathering evidence and beginning user testing was next. This was to ensure and determine if other people would notice the same subtle qualities within each. This would allow for an in-depth analysis and provide the concrete evidence as to why I decided what spatial plug-in to use.
Deahl, D. (2018). Google is making it easier to create 3D audio for VR. [online] The Verge. Available at: https://www.theverge.com/2017/11/6/16614348/google-resonance-audio-3d-sound-vr-ar [Accessed 23 Apr. 2018].
Google Developers. (2018). Developer Guide for Resonance Audio for Unity | Resonance Audio | Google Developers. [online] Available at: https://developers.google.com/resonance-audio/develop/unity/developer-guide [Accessed 19 Apr. 2018].
Gould, R. (2018). Let’s Test: 3D Audio Spatialization Plugins. [online] Designingsound.org. Available at: http://designingsound.org/2018/03/29/lets-test-3d-audio-spatialization-plugins/ [Accessed 16 Apr. 2018].
Valvesoftware.github.io. (2018). Steam Audio Unity Plugin 2.0-beta.13. [online] Available at: https://valvesoftware.github.io/steam-audio/doc/phonon_unity.html [Accessed 23 Apr. 2018].