.jpeg)
Think about the last time you experienced a digital story. Was it a movie? A short video? Maybe the story mode of your favorite video game? Regardless, the primary way that digital stories have been told is through sight and sound, sometimes with an interactive element. However, with the advances in virtual reality over the past few years, the way we think about sight and sound is changing, and these changes are allowing us to experience digital stories in a whole new way.
The power of storytelling is the ability for a good story to invoke the emotions of the audience, resulting in the formation of a connection with the narrative and characters. VR allows a storyteller to enhance the engagement of their audience through immersive storytelling.
When you move your arm, head, and body in a certain way, your character copies your movement.
Immersive storytelling allows you to step into the shoes of your favorite characters. Instead of experiencing a story as an audience you can actually step into other people's lives. This process works by inhibiting some of your sensory abilities and replacing it with intentional sensory stimuli that aids in telling a richer story. Most virtual reality equipment that is on the market for consumers consists of vision goggles, headphones and in some cases mobile handheld controllers.
Visual storytelling has been transformed into a fully immersive experience. VR does this in two ways. The first is by cutting off the sight of reality, and the second by offering a virtual replacement. The first part, blocking out reality, seems fairly self-explanatory so I’ll move on.
The second part is where things get cool.
The view that you get of the virtual environment with most VR goggles on the market is about 110 degrees horizontally, which is equivalent to that of the human eye (not including peripheral vision). By replacing your sight with a virtual sight with similar dimensions to what you would naturally be seeing, it adds to the feeling of you actually seeing something first hand as opposed to watching something take place.
The speed at which the image can process also adds to this feeling. It would be really disorienting if there was a lag time between your head movement and the image you are seeing. In order to avoid this, it means that the VRs frame rate needs to be quick enough to trick you into thinking the image moved simultaneously with your body movement. To put in perspective just how fast that needs to be, the brain can process an image in about 13 milliseconds. In order to even come close to providing natural sight movement a VR image needs to respond to body movement in about 50 milliseconds, although some VRs can do it in less. The speed at which the image can respond is also limited by the need to produce clear crisp visuals, not blurry ones.
Sound plays a big role in the art of storytelling. Whether it’s through music, dialogue, or background noises, no one can deny that sound adds a lot to the story experience. VR uses sound as a method to enhance the interactive visuals and object orientation. Sound in VR is just as important as the visual aspect because as humans we orient ourselves a lot through our sense of hearing.
Advancements in audio technology such as Spatial Sound allows for the creation of totally artificial sonic environments
Audio technology for virtual reality environments has rapidly been improving to the point where it is pretty similar to how we naturally hear. Especially in terms of being able to sense the direction the sound is coming from, how far away an object is and how different things sound differently as we interact with them. All of this sounds pretty basic when comparing it to how we naturally hear, but when you think about how all of that is done through just two headphones, it’s pretty impressive.
The way sound is developed for VR is different from how sound is used in videos or non-VR video games. Whatever you choose to look at or do in your virtual environment the sounds will match up as if it were reality. The audio function in a virtual reality space can track your head movement and place sounds 360 degrees around you. Because of this VR audio can alert a user to something that’s not in their line of sight in a directionally accurate way, including above or below you. This means that just through headphones you can hear directionally like you would naturally. Developers can also control how sound moves as the object causing the sound moves closer or further away from the user in virtual space and caters this to each individual object sound. Not only can the sounds change as you move around the virtual space, but they change depending on the object. This works in tandem with the visual aspect of VR to make the overall experience more engaging and realistic.
Some VR platforms also employ an interactive element that can enhance the experience. You are no longer just able to imagine yourself walking in someone else’s shoes, but actually be in the shoes the virtual world gives you. Handheld controllers for some VR video games allow the audience to actually participate in the storyline or video. With the innovation of VR technology, app creativity to compliment the technology has quietly followed.
The natural field of vision paired with the 360 sound capabilities and interactive controllers gives you a true 3-dimensional experience that works similar to your natural senses. The audio and image technology have moved VR forward to the point where it feels less like playing a game or watching a video and more like being transported to another world.
The senses in storytelling that VR technology currently on the market can influence are limited to sight and sound. If you are interested in where the VR industry is headed in terms of controlling aspects of sensory input like smell and kinetic movement check back next week to read more about the future of VR.
No Comments Yet
Let us know what you think