In September, Disney invested over $60 million in the Virtual Reality start-up company Jaunt, another sign that the once-dormant format was back on the upswing. It wasn’t too long ago that VR was considered a joke, a relic from the 90s as kitschy and outdated as The Lawnmower Man. But the VR movement has now come back full circle into the limelight. Last year, ILM released the ‘Trials on Tatooine’, a short VR piece in conjunction with Star Wars: The Force Awakens and just this month, Disney released two short The Jungle Book VR experiences. In King Louie: Cold Lairs, you see through Mowgli’s eyes as he comes face to face with titular Great Ape (you can view said VR experience here).
Last week, I spoke with Keith Miller, the Visual Effects Supervisor for WETA Digital, who oversaw the creation of Cold Lairs. Miller’s been working in virtual reality for over a decade, co-founding Applied Interactive (which brought VR into the contemporary art scene) and now using these same techniques to create one minute VR pieces for WETA. In the following interview with Miller, he discusses the process of crafting a VR world for The Jungle Book, the limitations of the technology and its possible and inevitable growth.
How early in the making of The Jungle Book, is the idea of a virtual reality experience brought up?
KEITH MILLER: We were in the home stretch of our regular VFX work for the film last year. Sometime around October, we started hearing whispers of a VR project and those conversations grew. Eventually we got approved to do a test - just a single frame test – to put the user inside the space of the temple, King Louie's throne room.
What is that single frame test?
MILLER: It's basically a 360° rendering of King Louie's throne room environment.
How do you determine which scene to focus on for the virtual reality experience?
MILLER: We had one main environment from the bulk of our VFX work: a full CG environment for the temple room area. We needed it to be a full CG environment so that we could drop camera positions throughout. We needed to have coverage everywhere… so [the temple room] made a lot of sense as a starting point. And even beyond that, cinematically - it's really beautiful. It helps you to understand the scale of King Louie and the scale of that environment when you're looking at things through a child's eyes, from Mowgli's height…
Was that always the idea to have it through Mowgli's eyes?
MILLER: The idea was always that it would be Mowgli's point of view inside that environment. We wanted to recycle some of the extensive animation that we already had for the film. We compress[ed] a series of shots from that environment that are then stitched together with filled in animation to bridge into one cohesive unified sixty-second piece.
I was going to ask if there were storyboards but instead you’re using VFX you've already made…
MILLER: Exactly. There's definitely a large amount of new animation in there in terms of the actual environment, lighting, characters. All of that is an extension of what we had already done…
How do you determine where the story is placed within the 360° environment?
MILLER: That's part of the exploration for us. It's a completely new process in terms of the animation and how you draw the viewers’ eyes to certain things. Gone is the idea of having a character leave frame. You don't have a two dimensional window into that world… We start the piece out with the viewer facing the entrance of the monkeys and King Louie is behind you. That was a conscious decision to keep that moment when Louie first speaks as a bit of a surprise. We do this thing where we have this monkey jump down from the column right in front of you and move around to your left and jump back behind you around the column. The intent is that it would catch your eye and you would follow that monkey around right as Louie begins to speak. So you [need] clever little things like that to help draw the viewers’ eyes into certain areas. You can watch the piece ten, twenty times but you're still not going to necessarily catch all the details in there which makes the review process very difficult. As a video tech facility, our standard practice and review processes are based around a camera as a window...
I was going to ask how is the process different here from standard VFX shots?
MILLER: In a regular VFX shot for a film, you have a carefully crafted composition and camera move. So that's completely gone now. You're rendering things through a pinhole camera so instead of that small window you're looking at that entire world. You can't be just concerned with what's happening in one particular area or one particular direction. You have to be looking at what's happening behind you, what's happening off to your left, off to your right and you're animating and working with an entire world rather than this little tiny slice of action.
What is the ‘uncanny valley’ for animating animals on a VR project?
MILLER: For us on this particular piece, it's a very photo-real environment. We spent a lot of time crafting the details and then spending time with King Louie and the monkeys around him. We've had a lot of experience with simians [on The Planet of the Apes films]. You very quickly make the leap of faith getting past ‘Oh this is a talking animal’ to the idea that this is just a character in the film. If you put that attention of detail to the characters, you can quickly relate to them and understand how they emote and express things. Once you're getting into humans, digital humans, it's a different ballgame, similar process to a degree but it's certainly a much more tricky area and you're much more susceptible to getting into that ‘uncanny valley’ area but that wasn't a concern for this piece.
How long does it take to make a VR piece?
MILLER: We got the green light at the beginning of this year - probably January and at that point, we hit the ground running to get things done in time. We had to have this done by the very first week of April - so we had three months to pull it all together. We had used the single frame test from last year to push through all the technological aspects so that once we were able to get the green light we were really able to dig right in. But at that point – it’s more rendering that needs to get done. The rendering for this stuff is pretty huge. I think we rendered a solid month for a sixty-second piece. And we took a lot of things you wouldn't notice necessarily - but there are a lot of optimizations and shortcuts to make things efficient. For example the environment itself is just a single frame render and then we rendered the monkey shadows and all their contacts per frame. Then that gets composited into the single frame environment.
How do you feel Virtual Reality technology has grown?
MILLER: It's been nearly twenty years since I first started working in the VR scene. Back then we were working in what is effectively a million dollar VR system. You're in a room-size rear-projection system and it was so expensive that those environments are restricted to high-end educational institutions, research type settings. At the time, we formed the non-profit Applied Interactive. The idea was to push that technology into the hands of the artist and people who want to build and explore. What we did back then was create portable systems. [They] were single screen rear-projection systems that we could travel around with and install and sell and build consoles for. You could build them at the time for ten or twenty thousand dollars. At the time being a portable system was fantastic. I think that's one of the things I find so impressive with the state of the technology today especially something like the Samsung Gear and these phone driven technologies that don't require separate desktop settings. It's amazing the portability of these systems. I can carry a [VR] headset around from meeting to meeting. It's just mind-blowing.
What made you get in Virtual Reality to begin with?
MILLER: I’ve always been fascinated with the fusion of art and technology. I remember experimenting on a Commodore as a kid. I had an old KoalaPad – one of these pretty primitive digital tablets. I always kept one foot in each of the scenes - art and technology. As I got into undergrad and university, I moved my way through architecture and industrial design and computer graphics; but all at the same time I was keeping a foot in the technological side, studying computer science and mathematics. I think it's one of those things that carried through. When I was researching graduate schools, I hadn't even really dabbled or heard much about the VR scene but I was researching the electronic simulation laboratory there and the concept of what they were doing: this hybrid program of computer science and artistry really resonated.
It seems that VR’s become much more popular in the past couple years…
MILLER: We'll continue to see a blurring of the lines between categories of VR. There's a lot things being called VR now. You've got 180° Monoscopic movies. You've got 360° movie content. You've got 360° Stereoscopic content which is what our piece is. I think as the technology improves we'll start to see a blurring of the line between these things. I think the other thing too - the quality of the content is going to become dramatically better. At the moment, everybody is still really exploring and understanding how to tell a story, how to create narrative content in a way that exploits the medium and produces something compelling enough that people are willing to pay for. There's a lot of really bad content out there and there's some really intriguing things as well. I think as that language gets established people are going to become comfortable with content [and] willing to pay. Because all that creative stuff at the moment is free. The only people who have figured out how to monetize [VR] are mostly on the game industry side. So I think it will be similar to the explosion of streaming content and those non-standard distribution models. It's just going to continue to explode on the gross side.
Is it possible to do a feature VR film?
MILLER: I think it's coming. Yeah - it could be time consuming as a 3D render of a 360° piece. There's a lot of live action content that people are creating as well. I think the limitations at the moment are possibly the ergonomics & comfort of wearing the headset and being immersed in the world for a long period of time. I think for the moment, people are spending one, two, five, ten minutes a time inside virtual environments. You can get susceptible to a bit of eyestrain. Beyond the ergonomics of the hardware and what the experience of staring at a screen two inches from your eye for half an hour or an hour or two hours, how do you make content that is compelling to watch for two hours?
The Jungle Book is currently playing in theaters everywhere.
