The majority of influential philosophers over the course of history, even some relatively modern ones, wouldn’t have believed you if you described virtual reality (VR) to them. If they had been around to see it, however, they may have pondered how their principles could be applied to this new technology. The science behind psychoacoustic principles as they relate to the listening experience in (VR) has already been explored, so today we will dive into the philosophical principles that affect listening in this new medium. When we quote George Lucas and say that “sound is half the experience,” we are borrowing his acclaim to emphasize the importance of sound. Sound is at least that important and is often neglected more than the veggie platter at Thanksgiving dinner. With this gravity in mind, this article will focus on how you experience that sound.
The principal agent of experience is a person, so it follows that every experience is very personal. An individual’s experience is made up of personal perception, thoughts, emotions, and beliefs. Just designing experiences for physical consumer products like smartphones isn’t easy — imagine how hard designing a great VR experience is!
John Dewey, who is known for exploring the philosophy of pragmatism, proposed that experience is based on two principles. The first concept of continuity explains that all experiences affect future experiences, for better or for worse. Experiences, therefore, are not independent of each other nor are they one-time events — each one has a relationship to something that happened in the past or will affect something in the future. For example, the fact that you are finally watching a VR piece is related to the fact that you recently got a membership for a VR content subscription service like Wevr’s Transport VR. Or maybe your enjoyable experience of The Mummy VR Zero Gravity Stunt Experience inspired you to go watch The Mummy movie in theaters.
Within this continuity framework, the VR listening experience has the potential to be very confusing. The way you hear sounds in VR might be totally different from the way you have been listening in the real world or while consuming more traditional content. At a concert or a conference, you see where the sound is originally generated from — the musician or the lecturer. However, unless you’re a VIP every time, you are actually hearing the sound from loudspeaker locations and not from the actual sound sources. This discrepancy also occurs when you watch something at the theater. The screen, filled with things that are responsible for the sound, is placed at the front while the sound is actually projected from speakers in various locations. Fundamentally, however, people are familiar with this listening experience — visuals are in front of them and sounds don’t necessarily match what they’re seeing.
VR finally frees us from the constraints of physical loudspeaker placement. The virtual speakers that replace physical ones can be unlimited in number and sound objects can be placed anywhere in a 3D space without the need to map sounds to specific channel formats. Thanks to advanced spatial audio and binaural rendering solutions, VR headset users can hear sounds that precisely match their visual location. This simple principle — that sound can finally come from where it is actually located — can confuse people. Ironically, the accuracy might feel unnatural at first and it can even cause a negative response from some users. Our familiarity with previous listening experiences is responsible for generating this reaction. Nobody knows for sure what our future listening experience will be like, but we can bet it will be shaped somehow by what we are doing now.
The second principle of interaction may provide some clarity. The interaction between environmental factors and a person contribute to experience in a profound way. For example, the experience of wearing a Gear VR is made up of the interaction between you, the Gear VR, and the situation that the Gear VR has put you in. Interactions occur when humans are driven to behave in a certain way. Those actions then cause a reaction in the environment. We internalize those environmental changes and they shape our experiences in the future. In other words, the entirety of an interaction involves a person actively doing something, internalizing the environmental reaction, and then recognizing this relationship as a whole.
Humans can’t just interact in a void — they need an environment and objects to interact with in order to drive the experiential cycle. Environments and objects don’t have to be perfect either since whatever form they take will allow us to shape experience from them. If a VR content creator experiments with various ways of crafting a listening experience, then each piece of content that is produced becomes an environment for a person to interact with. People will incorporate interactions from new types of VR content into their overall experience base, and these unfamiliar experiences will quickly become comfortable.
Familiar but false or strange but true
We build new knowledge through experiences and all of our current knowledge is based on the experiences that came before. Even our imagination, a realm that is not physically accessible in the real world, is influenced by previous experiences. The more experiences we have, the more creative we can try to be.
Consuming different styles and concepts of VR content will be the key to building our experience base moving forward. We might learn that positioning ambient sound to the sides is more comfortable than spreading it throughout the entire scene. We might learn that volume manipulation could be effectively used to help people adjust more easily to this relatively strange listening experience. Or the decision might still fall down to familiar but false or strange but true, and even then, only after thousands and thousands of experiences have been accumulated.
While the creative campaigns like those of the above certainly help to engage with consumers in new and interesting ways, the use of this technology in the business world shouldn’t stop there. It’s just the tip of the VR iceberg…
