Noah J Nelson on Friday, Sep. 5th
1. (of a computer display or system) generating a three-dimensional image that appears to surround the user. Source: Google.
Immersive. The word pops up in conversations about entertainment with as much frequency as “engagement.” While the definition is tied to its roots as techno-jargon in the cyberdelic 90s, its popularity comes from the fact that the meaning has grown beyond those roots.
Facebook’s acquisition of virtual reality start-up Oculus VR earlier this year put the word back in the mouths of the mainstream press, and this week Samsung announced the Gear VR head mounted display adapter for their next generation phone. Another use of the term is tied to immersive theater productions like the long-running Sleep No More in New York City.
Whether in virtual or flesh and blood reality, the singular goal of an immersive experience is to suspend disbelief so totally that the audience gets wrapped up in the world around them to the exclusion of any other.
What follows is a primer, of sorts, on what the future of immersive media will look like.
Virtual Reality is more than Games
Let’s start with the obvious: games are going to push the envelope on virtual reality like no other art form. The level of agency that games already give their audience and the diversity of development that is taking place in the game segment of VR surpasses any other field. Without the vigorous support of game enthusiasts the Oculus Rift wouldn’t have blown past its original crowdfunding goal and jumpstarted an industry.
What we can already see being made is far more than games.
Silicon Valley start-up JauntVR is developing a live-action camera that empowers filmmakers to create 360 degree experiences with live actors. One of the first efforts, produced by New Deal Studios, will be the WWII-themed The Mission VR which is slated for release close to the arrival of Samsung’s Gear VR. While audience members won’t have control over the narrative of The Mission VR they will have the power to look wherever they want to in the scene, which creates new storytelling challenges for the filmmakers.
Meanwhile Los Angeles-based journalist Nonny de la Pena’s immersive journalism work has turned heads at film festivals (Sundance, Tribeca), and points the way forward for new ways of telling nonfiction stories using video game engines to create settings that audiences can explore.
These two examples run at the extremes of what non-gaming immersive VR experiences can be. The virtual reality platforms themselves, however, will have content that hails from more traditional media. Not the least of which will be trailers for IMAX format movies shown in a virtual IMAX theater strapped to your face.
Immersives are more than VR
Even with all the emphasis on virtual reality the immersive movement in entertainment goes far beyond head mounted displays. (Here we could get into a round of academic navel gazing about what “virtual reality” means, but for the sake of argument let’s run with what the general consensus that VR means strapping a screen to your face.)
The London-based Punchdrunk, creators of Sleep No More, are part of a vanguard of theater artists who are breaking down the fourth wall and creating theatrical experiences that audiences can physically move through. While there are those who like to argue that immersive is a state of mind—an emotional shorthand for “really, truly engaging”—the audiences who flock to these productions are generally looking for one thing: to bodily step into another world.
There’s a lot more to this theatrical movement than New York and London productions. Theater artists around the world have begun to aggressively experiment with the form.
A different kind of theater has taken off in China, and has made landfall in the home of the movie industry: 4D theaters. These are movie theaters that are equipped with machines that bring smoke, rain, and other environmental effects to the audience, designed to synch up with the on-screen action the way a soundtrack is. Variety reported this week that attendance at the nation’s first dedicated 4D screen is handily beating regular screen averages.
While the experience can be as gimmicky as post-processed 3D, but when executed well can create its own kind of magic. There’s something to be said for snow falling during a screening of The Nightmare Before Christmas.
Games will define the way other immersive content is made
Define might be a little strong, but there is no escaping the fact that the influence of video games are going to be felt in the way that people approach immersive content. While some tools will allow for more interaction than others—a game engine can be freely explored while a JauntVR image is tethered to the camera’s point of view on the field of action—the logic of games and game narratives are being taken into consideration when immersive experiences are designed.
Punchdrunk’s Felix Barrett has explained his company’s work using examples from Skyrim and Gone Home two very different narrative experiences that themselves exist at the extremes of video games. Even when an experience is linear the 360-degree element of immersive film means that storytellers will be wise to look at how open-world games manage a player’s attention.
Alternatively a filmmaker can trigger a piece of video based on where the audience member is looking, which is itself similar to the scripted events found in some video games.
Media giants are already investing
Look at the list of major film studios who were announced as developing content for the Samsung Gear VR this week. Fox, Warner Bros., Paramount: these are not start-ups with a wild glint in their eye. These are the grasping hands of multinational corporations determined to rule the world. No, wait, that’s Cobra.
Still: it’s clear that the notoriously risk-averse studios are putting real dollars into making content for VR. What’s not as obvious is what that exactly means, or what else studios are doing in the realm of immersive entertainment.
The first wave of VR content coming out of the studios are promotional support for their traditional offerings. We saw that at Comic Con this year with experiences based on Sleepy Hollow and X-Men (Fox) and Pacific Rim (Legendary). Until the studios can figure out how to monetize VR this is probably what they will sink development assets into.
Once they figure out how to make people pay—which will probably be in the form of micropayments, yuck—then the sky is the limit on what we get. (I’m voting for a full X-Wing pilot life simulator and episodic content which will let me fulfill my dreams of being Batman, but that’s just because I’m in a heavy #normcore phase right now.)
On the live-action side of the immersive equation we have the theme park wizards behind Disneyland tinkering with live-action roleplaying game experiments in the parks, and searching for ways to take those experiences out into the “default world.”
That Burning Man inspired word choice isn’t an accident. As one friend returning from this year’s gathering at Black Rock City pointed out, Burning Man is one big immersive arts festival. We can find echoes of that kind of interactive, immersive art at music festivals like Coachella and Electric Daisy Carnival. While these events are not focused on immersive experiences, they have been acclimating audiences to the idea.
Innovation means more than technology
We tend to think of innovation in terms of technology, but it also happens in terms of technique. (If you want to be pedantic, the words have the same root.)
While innovation in VR is starting with major technical breakthroughs—low latency, high resolution screens, positional tracking—success for the medium means that there will be changes in the way that content is made.
Already the makers of The Mission VR are finding that 360 degree filmmaking means you have to carefully plan your action. Without simultaneous video playback—it takes 15 seconds to render one second of Jaunt footage—the director and cinematographer have to trust what the camera captures. In a way this pushes filmmaking back to an earlier era. While the speed of the immersive cameras will almost certainly accelerate to the point where near-simultaneous feedback will be possible, for now it all comes down to planning and a little bit of luck.
Actors in immersive films are also going to have to adapt the ways that they give performances, which leads us to our next item:
Immersion raises the stakes for engagement
Here’s an unavoidable fact: the more intimate a fictional experience is, the easier it is for bad acting to blow the whole deal.
The magic of immersive cinema comes from the fact that it is surprisingly easy to trick the brain into suspending disbelief when a turn of the head reveals the world just outside of view.(With little apparent lag, of course, otherwise deal’s off.) Now drop an uncommitted actor into this scenario.
Suddenly what was believable becomes dissonant. This is something I’ve seen in otherwise beautiful tech demos. Not that tech demos are where you expect to see brilliant acting, but it shocked me how much I was thrown out of an otherwise overwhelming experience by some shoddy make-up and an inauthentic actor.
We are going to find that bad acting and other obviously inauthentic elements in otherwise realistic worlds will stand out like a sore thumb. This is akin to the problems people reported with the high-frame rate projection of The Hobbit. While the New Zealand visits were incredible, some people described the make-up and sets as looking fake.
Actors fluent in stage and screen already know that they have to modify their performances based on the strengths and weaknesses of those mediums. It’s a long-held axiom that stage actors have an easier time translating to film than film actors to the stage. The idea being that it is simpler to “tone down” a performance for the screen than it is to blow one up for the stage. VR cinema is going to be a third way: neither stage nor screen, but something which will benefit from the subtlety of film acting and a stage actor’s stamina and presence.
Cross training is essential
Which bring us to our final note.
Whether we are talking about game makers exploring narrative, filmmakers experimenting with open worlds or theater artists wielding game mechanics, creators would be wise to use an interdisciplinary approach with VR.
Experience design has come into its own as a field in the Internet era, and this approach to designing engagement steals liberally from every artistic discipline known to humanity. Those who have a magpie mind—picking up tools from every corner—have an advantage in creating compelling experiences, which go beyond stories or artwork alone.
Creators who have a knack for collaboration, and bringing out the best in their teammates, have even more of an advantage. The real world, after all, is a collaborative project. The most compelling virtual worlds are no different.
Image: Elite Dangerous
Burning Man • Coachella • Electric Daisy Carnival • Facebook • fox • Gear VR • immersive • Jaunt VR • Legendary • New Deal Studios • Oculus Rift • Oculus VR • Punchdrunk • Samsung • Sleep No More • The Mission VR • Warner Bros.