With all of the focus landing on The Hobbit: An Unexpected Journey's use of high frame rate/48fps, the rest of the production's stunning achievements in visual effects have been unfairly shunted to the sidelines.  Thankfully, four-time Oscar/BAFTA/VES winner Joe Letteri recently spoke about the VFX behind the first film in Peter Jackson's new trilogy and we were invited to check out the presentation.  As both the visual effects supervisor on the films and the current Director of Weta Digital, Letteri pulled back the curtain on the unbelievable amount of work, creativity and expertise that goes into making a film like The Hobbit.  Hit the jump to watch a video featurette on the film's VFX, as well as a break down of Letteri's presentation with some incredibly cool behind-the-scenes information.

Check out the featurette "VFX of THE HOBBIT: Fantastical Creatures & Lands of Epic Beauty & Darkness," followed by highlights from Letteri's presentation, courtesy of USC's Institute of Creative Technologies and a meeting of the Virtual Production Committee:

joe-letteri-weta-digital

Letteri has been in the visual effects business since 1989's The Abyss, but really started to integrate virtual production with live action production on set with 2002's The Lord of the Rings: The Two Towers.  With Andy Serkis coming on board the picture as Gollum,  originally in a voice role only, but wanting to get more involved with the actual acting, the two brought an assistant director in to the motion-capture stage and pushed the boundaries of a then less-refined technology.  While there was no real-time facial capture used in the Lord of the Rings trilogy, it was done on Avatar with the addition of head rigs and on Rise of the Planet of the Apes, which marked the first time a virtual production was done side-by-side with a live-action production.

Now that we've seen where the virtual production technology has come from, let's take a look at the highlights of the most recent implementation of these tools with The Hobbit: An Unexpected Journey.  I'll follow Letteri's presentation and break the information down as he did:

the-hobbit-an-unexpected-journey-gollum-andy-serkis

Gollum:

  • The first scene shot for The Hobbit was the "Riddles in the Dark" scene between Serkis and Martin Freeman as Bilbo.
  • Although Serkis' eyeline, head attitude and hand gestures matched up with the virtual Gollum character, their body proportions were different, which required additional sculpting after the motion capture.
  • Gollum exists as a full 3D model in the scene and was not created from Serkis' performance against a green screen.
  • The high frame rate/48fps works to the creative advantage for refining micromovements in facial expression, but is a real pain for artists who have to paint and rotoscope twice as many frames as they're used to.
  • Gollum was rebuilt from the ground up for The Hobbit. 
  • There are more details in Gollum's face in The Hobbit than in his entire body in The Lord of the Rings. 
  • Weta Digital's process of building virtual biological creatures with complete skeletons and musculature makes a more realistic creature than even an animatronic puppet.

the-hobbit-trolls

Trolls:

  • Similar techniques used in bringing Gollum to life were also used for the troll scene.
  • A virtual production stage was set up next to the live action stage, with footage from both sets overlapped onto monitors and refined in the post-production process over time.

Bag End:

  • The scene of the dinner party at Bag End is a throwback to Gandalf's (Sir Ian McKellen) arrival at Bilbo's home, in which he famously (and accidentally) bumped his head on the chandelier.
  • Rather than use wholly practical forced perspective, Jackson wanted the camera to move around the actors and give them more freedom of movement, resulting in "dynamic forced perspective."
  • To achieve this, two interior sets were built: one to the scale of Bilbo and the Dwarves (ie normal) and one to show Gandalf in a cramped and confined space; the former set was dressed with the homey interior you see in the film, while the latter was fully draped in green screen.  One master camera followed a pre-determined track on the first set with a slave camera matching its motions on the green screen set.  The non-green screen camera was scaled up 33% to accommodate the scale change.
  • There is a four-frame delay on the slave cam, as well as a potentially dangerous trait of an increased speed ramp.  For this reason (and to keep McKellen from acting by himself in a green room all day), the Bag End scene was finished in only four takes!
  • The shot took 18 months to put together, however, owing to post-production animation that massaged the scenes together and blended them seamlessly.

hobbit-goblin

Goblins:

  • For everyone who laments the lack of practical effects, the team tried it...a lot!  One scenario involved people in goblin suits herding our heroes along a rope bridge.  Originally they had animatronic heads, which didn't last long.  Eventually they were replaced altogether by using 3D composites, rotoscoping the live-action elements via a disparity map and using 3D to integrate. <- From the mouth of Letteri, since I understand almost none of that.
  • Goblin motion was recreated on a motion-capture stage in order to line-up with the shot.  Believe it or not, Dwalin (Graham McTavish) was actually digitally replaced in this scene as well, since it was easier to replace everyone than build around one live-action character.  Keep an eye out for this scene when you watch it again and see if you can spot the swap.
  • Motion capture was also used to create the sword fight between a goblin and Bilbo, though the goblin fighter was later scaled down in post-production.  Bilbo's face is actually digitally replaced since a stunt man was used in place of Freeman.
  • The shots of the goblin cavern were mostly digital, with only a few practical builds used.  These were shot mostly for reference and to aid in the actors' performances, but were later replaced digitally.
  • The swinging rope bridge scene was entirely digital except for the eventual rope cut.  The animators themselves actually donned motion-capture suits to simulate our heroes as they stumbled and jumped off the swinging bridge...it took them a few tries.

great-goblin

Goblin King/Great Goblin:

  • Peter Jackson's note to the creative team was to "make him the most disgusting thing you can think of."  I'd say they succeeded.
  • Barry Humphries performed both on the motion capture stage and re-recording dialogue with the use of a head rig to capture facial animation.

Azog:

  • The original incarnation of Azog was, again, a practical suit with a live actor.  The result looked like a man-sized molted porcupine, so they went with a brand new digital creation.  (The "molted porcupine" became another character in the film, which you may be able to spot upon another viewing.)
  • The new Azog was created in about six weeks.

Radagast the Brown: 

  • The chase scene for Radagast (Sylvester McCoy) was also originally done practically, using a small vehicle to drag the sled through a practically built forest.  The set helped McCoy's performance and was used as a reference, but the only thing that was kept from the shot was McCoy himself.  Everything else was replaced digitally.
  • Since they replaced a majority of the landscape with digital creations, the programmers wrote completely new software, dubbed LumberJack, which not only grows and animates trees from the ground up, but it gives the animators dynamic control down to the level of pine needles.  The software took about a year to write.

THE HOBBIT: AN UNEXPECTED JOURNEY - rivendell

Rivendell:

  • Letteri showed a scene in which Bilbo and the Dwarves walk across a bridge into Rivendell.  The only problem with the green screen shot was that the ceiling and skylights were not part of the green screen.  The animators solved this by chopping out the ceiling...as well as the actors' hair!  Another bit of software, named BarberShop, allowed the animators to grow and style digital hair for each actor, which also made replacing the transitional areas between live-action and green screen more seamless.
  • There were no miniatures used in The Hobbit due to scheduling issues and the fact that the pre-existing miniatures were too low definition, even though they scanned them for references.  Almost everything was built digitally, from the waterfalls to the birds (each of which had their own skeletons and musculature built in).  Some sets were built practically and then expanded digitally if Jackson wanted to go wider on a particular shot.

ian-mckellen

Q&A:

  • Letteri's wishlist for the next level of technology includes faster rendering time to eliminate lag and provide real-time playback of digital creations.
  • Letteri stressed that 48fps is a distinction of temporal resolution and not spatial resolution. A single frame pulled from a 24fps movie will look the same as one pulled from a 48fps movie; the latter simply has more frames in the same amount of time.
  • Letteri also commented that perception of stereo (what we call 3D) is independent of the high frame rate.
  • They chose to go with 48fps instead of 60fps because the distinction between 24fps and 48fps was significant, while the difference between 48fps and 60fps was not as apparent.  Also, they presumed that exhibitors would more easily be able to adapt to the 48fps format.
  • The cameras used a 270° shutter angle to compromise between 24fps and 48fps. <- Again, from Letteri, not me.
  • One member in attendance pointed out that The Hobbit essentially used their 1st unit cast to do a live-action pre-viz for each scene, which was later referenced and replaced using digital techniques.
  • Few scenes had to be post-converted from 2D to 3D.  One such scene was a closeup of Bilbo early on, due to a failure in one of the cameras.
  • Letteri said he's experienced no lag filming in 3D, after a brief initial learning curve.

The Hobbit: An Unexpected Journey is up for Best Achievement in Makeup and Hairstyling, Best Achievement in Production Design and, of course, Best Achievement in Visual Effects at the 85th Annual Academy Awards on February 24th.

The Hobbit: An Unexpected Journey will arrive on Digital Download on March 12th and on 3D Blu-ray Combo Pack, Blu-ray Combo Pack and 2-Disc DVD Special Edition on March 19th. 

martin-freeman-the-hobbit-an-unexpected-journey-poster