What happens to a film's story when blades of grass are more interesting than the plot? When you can lean in so close to an actor you ought to feel their breath? When a few simple steps around a room can make you dizzy?
Virtual reality filmmakers are grappling with these challenges as they explore a medium that hasn't had a century to develop the way movies have. Much the way early cinematographers had to discover cuts, wide shots and perspective, VR filmmakers find themselves making things up as they go along, often mashing up traditional movies and video-game techniques along the way.
Consider for a second how we've all internalized the visual language of film. Quick cuts skip over time and space to get to a story's point. Wide shots give us a godlike perspective. Zooms and tight focus naturally draw our attention to specific points. Audiences have been trained to stitch it all together into a fluid narrative.
Now picture yourself in the middle of the action. With a headset strapped over your eyes and ears, sometimes with a controller in your hand, you are completely immersed in the virtual world. Things can happen behind you. There are characters and places to explore from different angles.
It's all a work in progress, one that more consumers will see when the long awaited Oculus Rift VR headset ships later in March. It's crucial for VR filmmakers to get storytelling right, or the whole virtual reality boom could turn out to be just another sandbox for hardcore video gamers .
JUST LET GO
This was one of the first lessons for Oculus Story Studio, a hub for digital animators and experimental filmmakers in San Francisco's trendy South of Market district.
In an early version of the Oculus short "Lost," audiences got distracted by looking at foliage or the moon and didn't pay attention to a giant, disembodied hand that's the subject of the piece, says creative director Saschka Unseld. So creators added a full 40 seconds to let viewers get oriented, and then just let go.
"Once you have the audience interested, they will look wherever the story happens," Unseld says. "But losing that control of focus of the audience is something that took a while to get over."
SUBTLE CUES DRIVE THE ACTION
In "Henry," another Oculus short, you find yourself standing inside a lonely hedgehog's computer-generated house. You're free to look around at anything, but it's not until you set eyes on the kitchen door that Henry shows up.
"That's a triggered moment that makes you feel like you discovered Henry in a very natural way," says director Ramiro Lopez Dau. Some viewers try to hug Henry, Dau says – but he doesn't hug back.
LET THE VIEWER DISCOVER WHO THEY ARE
At Montreal-based Felix & Paul Studios, filmmakers have exploded the traditional separation between film and filmgoer by making "Who am I?" a central question – sometimes even a mystery – for VR participants.
In the short film "Strangers with Patrick Watson," for example, a man riffs on a piano and sings. From a seat in the player's cluttered apartment, you can look at his wall art, peruse his collectibles and even watch his dog.
In Felix & Paul's "Jurassic World: Apatosaurus," you start off in a forest next to a jeep; a coffee mug and two-way radio lay within arm's reach. Turns out you're a park ranger visiting a sleeping diplodocid. That helps explain why the dinosaur just sniffs you in a familiar way and goes back to its daily routine chewing leaves.
A short based on the movie "Wild" takes things to a surreal new level. As her omniscient voice narrates, actress Reese Witherspoon hikes up and sits on a rock beside you. Then she looks right through you, and suddenly Laura Dern, playing Witherspoon's deceased mother, makes you turn to the right. Dern then talks through you to her daughter as if you're the ghost. Later, a curious fox comes up to sniff you, a reminder that you that you do, in fact, exist – virtually, at least.
"We don't ask people to come out of those things and say, 'I completely figured it out'," says Felix Lajeunesse, the studio's creative director.
SUDDEN MOVEMENT = NAUSEA
Vrse, a Los Angeles-based virtual reality studio that has worked with The New York Times, Vice News and others, has a signature documentary style involving a voice-over and stationary VR camera placement right in the middle of the action.
But it unhooked the camera in "Millions March NYC ," an 8-minute VR documentary on racial-profiling protests that Vrse produced with Vice News. Creative director Chris Milk acknowledged the hand-held camera and bouncing, shaky movements make the viewer "a little queasy."
It still proved a point. Turns out it's not motion so much as the acceleration and deceleration that makes people feel sick – the same way taking off and landing in a plane feel a lot worse than just cruising at 30,000 feet.
Many virtual reality filmmakers avoid motion because it induces nausea, but Milk says injecting a touch of acceleration is a new way for directors to provoke an emotional response. "You couldn't do this in a traditional, rectangular film," he says.
WHAT IS NARRATIVE, ANYWAY?
Can a story be an exploration of a space? Can it be just a moment in time and a way to understand a situation's context?
Creator Mac Cauley wrestled with these questions in "The Night Cafe," a scene painted in the impressionist style of Vincent Van Gogh. A trackpad lets you move around. Turn a corner and there's Van Gogh, listening to a piano player. A future version may include characters that react to your movement and talk back.
"Each person's going to have their own experience of it," Cauley said. "That's what makes the interactive medium of virtual reality so personal."
AP Video journalist Haven Daley in San Francisco contributed to this report.
