Why 48 Frames Per Second Is the Future of Filmmaking (Probably, If We Let It)

     January 1, 2013

the hobbit hfr filmstrip

If you are at Collider, you probably watch a lot of movies.  Based on our demographics, most of you have seen The Hobbit.  A significant portion of you will see The Hobbit in 48 frames per second.  Virtually all of you who do will think the higher frame rate looks strange, at least at first.  And yet, Hobbit director Peter Jackson proclaims 48fps is the future of filmmaking.  Critics are far from convinced, calling the new look “a gaudy high-definition tourist attraction” and “washed out and flat, yet unforgiving in its hyper-realism.”

Jackson’s push for 48fps (also known as High Frame Rate or HFR) has sparked a surprisingly heated debate over what seems like a relatively simple technological innovation.  Although my first viewing of The Hobbit was a peculiar experience, I am a believer in 48fps.  So after the jump, I examine the arguments for and against 48fps, the neuroscience behind the negative response, and what it will take for HFR to find widespread acceptance.

 

The Most Outrageous Thing

Before we start, I recommend you listen to this excerpt from the “Musical Language” episode of Radiolab that examines the arc of Igor Stravinsky’s The Rite of Spring from “being the most outrageous thing that literally maddens people, to a triumph, to kids music.”

A brief recap: Stravinsky premiered The Rite of Spring as a ballet in Paris in May 1913.  Stravinsky’s composition sounded different than the traditionally beautiful music of Wagner that this audience was used to.  In particular, Stravinsky relied on one dissonant chord, pounding it over and over throughout the entire piece.  The audience hated it.  They grew restless, started shouting and fighting each other, and this eventually escalated into a full-fledged riot.  Rioting cannot be a mere expression of disapproval for this crowd.  Something else must be going on here.

rite-of-spring ballet dancersScience writer Jonah Lehrer explains, “There are groups of neurons whose sole job it is to turn that dissonant note, dissect it, take it apart, and try to understand it.” Radiolab proposes that on that night, in the face of the repetitive dissonance of The Rite of Spring, those neurons failed to make sense of what they heard.  This failure to recognize the pattern changed the brain chemistry (specifically dopamine levels) in such a way that the audience temporarily went a bit crazy.

However, by the second run of shows in April 1914, audiences came out in droves—partly to see what this music was that caused a riot no doubt—and loved it.  They gave it a standing ovation.  The Rite of Spring became a part of the classic cannon, and just two decades later was palatable enough to feature in the Disney kids movie Fantasia.  Lehrer calls this “the perfect evidence of the brain’s astonishing plasticity.”

Our brains were presented with new information.  It was jarring at first.  We hated it.  Then, with context, we accepted the new information and learned to value it.  I believe 48fps can follow the same path, from controversy to natural innovation.  But we do have a say in this.

 

Why Increase the Frame Rate?

film stripFilm simulates fluid motion by capturing and projecting individual still frames of that motion.  Early silent films were projected at about 16-24fps until the frame rate was standardized at 24fps with the advent of sound film.  Filmmakers argued for higher frame rates to better capture motion and achieve decent sound quality.  Buyers pushed for lower frame rates, because a higher frame rate means more film, and film stock was expensive.  The result was a compromise at 24fps, which has been sufficient for the last century, but not necessarily optimal.  Heck, motion picture pioneer Thomas Edison himself reportedly recommended 46 frames per second because “anything less will strain the eye.”

Douglas Trumbull started developing Showscan, which uses 60fps, in the late 1970s.  The technology is in use at theme parks (Star Tours at Disneyland and the new King Kong attraction at Universal Studios use 60fps), but never caught on in cinemas.  James Cameron started touting the benefits of a higher frame rate as early as 2008:

“For three-fourths of a century of 2-D cinema, we have grown accustomed to the strobing effect produced by the 24 frame per second display rate.  When we see the same thing in 3-D, it stands out more, not because it is intrinsically worse, but because all other things have gotten better.  Suddenly the image looks so real it’s like you’re standing there in the room with the characters, but when the camera pans, there is this strange motion artifact.  It’s like you never saw it before, when in fact it’s been hiding in plain sight the whole time.  Some people call it judder, others strobing.  I call it annoying.  It’s also easily fixed, because the stereo renaissance is enabled by digital cinema, and digital cinema supplies the answer to the strobing problem.”

James_Cameron_Pace_Camera_G4 (1)

Cameron was especially frustrated because, even pre-Avatar, we had the technological capability to make a relatively easy switch to 48fps:

“Increasing the data-handling capacity of the projectors and servers is not a big deal, if there is demand.  I’ve run tests on 48 frame per second stereo and it is stunning.  The cameras can do it, the projectors can (with a small modification) do it.  So why aren’t we doing it, as an industry?”

Cameron has already stated his intention to shoot the Avatar sequels at 48fps.  In the meantime Peter Jackson—one of the few tech-minded filmmakers with nearly as much power in the industry as Cameron—took the lead on the battle for 48fps.  Here is how Jackson pitched the higher frame rate in a Facebook post he wrote while shooting The Hobbit in April 2011:

“The result looks like normal speed, but the image has hugely enhanced clarity and smoothness.  Looking at 24 frames every second may seem ok—and we’ve all seen thousands of films like this over the last 90 years—but there is often quite a lot of blur in each frame, during fast movements, and if the camera is moving around quickly, the image can judder or ‘strobe.’

Shooting and projecting at 48fps does a lot to get rid of these issues.  It looks much more lifelike, and it is much easier to watch, especially in 3-D.  We’ve been watching HOBBIT tests and dailies at 48fps now for several months, and we often sit through two hours worth of footage without getting any eye strain from the 3-D.”

Sounds like a pretty basic and natural technological advancement.  So what’s the problem?

 

What the Critics Say

Here are excerpts from a few of the critical reviews.

Todd McCarthy, THR:

the-hobbit-an-unexpected-journey-martin-freeman

“The print shown at Warner Bros. in what is being called ‘high frame rate 3D,’ while striking in some of the big spectacle scenes, predominantly looked like ultra-vivid television video, paradoxically lending the film an oddly theatrical look, especially in the cramped interior scenes in Bilbo Baggins’ home.  For its part, the 24fps 3D version had a softer, noticeably more textured image quality.”

Kenneth Turan, LA Times:

“Despite its drawbacks, The Hobbit, as noted, does have real virtues, and the best way to appreciate them is to see the movie, whether in 2-D or 3-D, in the traditional 24 frames per second format.  Though Jackson and other zealots for high frame rate would have you believe that the new system is more immersive, the truth is just the opposite.  Whatever its virtues may be from a technical point of view, audiences looking for a rich, textured, cinematic experience will be put off and disconcerted by an image that looks more like an advanced version of high definition television than a traditional movie.”

A.O.  Scott, New York Times:

peter jackson hobbit camera department

“Over all, though, the shiny hyper-reality robs Middle-earth of some of its misty, archaic atmosphere, turning it into a gaudy high-definition tourist attraction.”

Peter Travers, Rolling Stone:

“Couple that with 3D and the movie looks so hyper-real that you see everything that’s fake about it, from painted sets to prosthetic noses.  The unpleasant effect is similar to watching a movie on a new HD home-theater monitor, shadows obliterated by blinding light like—yikes!—reality TV.”

Peter Debruge, Variety:

“More disconcerting is the introduction of the film’s 48-frames-per-second digital cinematography, which solves the inherent stuttering effect of celluloid that occurs whenever a camera pans or horizontal movement crosses the frame—but at too great a cost.  Consequently, everything takes on an overblown, artificial quality in which the phoniness of the sets and costumes becomes obvious, while well-lit areas bleed into their surroundings, like watching a high-end home movie.”

Richard Corliss, Time:

hobbit-an-unexpected-journey

“Doubling the rate keeps the image from blurring when the camera moves, which is ideal for Jackson’s action sequences but disorienting in the more static interior scenes, where the scenery upstages the characters.  The clarity of the image is sometimes magical, occasionally migraine-inducing… At first, in the Smaug battle, I thought I was watching a video game: pellucid pictures of indistinct creatures.  After a while my eyes adjusted, as to a new pair of glasses, but it was still like watching a very expensively mounted live TV show on the world’s largest home TV screen.”

Dana Stevens, Slate:

“The best way I can think to describe the quality of the 48fps image in The Hobbit is this: It looks like an ’80s-era home video shot by someone who happened to be standing around on set while The Hobbit was being filmed.  (Other visual analogues scribbled down in my screening notes include Teletubbies and daytime soap operas.) The effect is curiously washed out and flat, yet unforgiving in its hyper-realism: Any imperfection or note of artifice in the costumes or sets stands out as if illuminated with a bank of fluorescent bulbs.  This wildly expensive visual technology paradoxically conspires to make everything else in the film look cheap.”

Drew McWeeny, Hitfix:

hobbit peter jackson slate

“I’m half-convinced that there was a projection problem when I saw the film, because I have trouble believing that what I saw reflected the desires of Peter Jackson and his team.  Throughout the entire film, there was a strange Benny Hill quality to sequences, with things that appeared to be sped up.  It happened in both dialogue and action sequences, and the overall effect was like watching the most beautifully mastered Blu-ray ever played at 1.5x speed.  It doesn’t make any sense to me that this process, which is supposedly all about clarity and resolution, would create that hyper-speedy quality unless they were doing something wrong in the projection of it.  Peter Jackson would see this immediately.  The voices are off-pitch, and the pacing of scenes goes to hell when it’s played this way.”

Matt Goldberg, Collider:

“Billed as a technology to sharpen 3D and reduce the headaches it can cause, HFR 3D has crippled Peter Jackson‘s return to Middle-earth.  Without the atrocious visuals, Jackson’s film is still slightly repetitive and bloated, but the magic mostly remains intact.  But under HFR 3D, the journey looks like a cheap soap opera on fast forward with crappy digital effects.”

I think that’s enough for now, but you can read other HFR-centric review roundups at Vulture and Badass Digest.  We can broadly separate the criticisms into two categories: “It looks like cheap video” and “It looks too realistic.”

Head to page 2 for more on the “soap opera effect,” how fantasy can use HFR, and why we can and should get used to 48fps.

Page 2

The Soap Opera Effect

“It felt like watching daytime soaps in HD, terrible BBC broadcasts, or Faerie Tale Theater circa 1985, only in amazingly sharp clarity and with hobbits.” -Jen Yamato, Movieline

In addition to the frame rate, we must also consider the flicker rate or refresh rate.  When projecting film, the projector closes the shutter (“flickers”) each time it pulls down a new frame of film.  So a projection of 24fps needs to flicker at least 24 times per second (Hz).  Most modern projectors already project 24fps at 48hz, meaning the shutter opens and closes twice for each new frame.  Some even go up to 72hz (3 flickers per frame) or higher.  This is done to exceed the critical flicker fusion rate (CFF) of the viewer to make sure that the light appears steady.  (Think of a 60hz light bulb.  The bulb does not emit a steady stream of light.  Rather, the bulb flickers at different light levels, but 60hz is fast enough so we only see the light, not the flicker.)

motion interpolation exampleThe refresh rate has become a key selling point for modern televisions.  The base level is 60hz, refreshing 60 times per second, but newer TVs go up to 120hz, 240hz, or even 480hz.  Often video is interlaced, separating each frame into one field of the odd lines and another field of the even.  So a 60hz television displaying interlaced video at 30fps refreshes each field 30 times per second.  This process uses up all 60 refreshes, but the dreaded “soap opera effect” arises with a higher refresh rate.

Televisions with a refresh rate of 120hz or greater are equipped with a “motion interpolation” setting.  (Your TV may call it something like TruMotion or SmoothMotion.) Rather than repeat frames, the television takes two existing frames, averages the brightness and color coordinates of each frame, and generates a new frame to insert in between the two frames.  For those who notice the effect, it looks unnatural to your eye because it is unnatural.  A computer algorithm approximates what the intermediate frame would look like if it were captured in the existing footage, and thus adds information to the image that is not entirely true to life.  This is especially true with quick camera movements, where the difference between two consecutive frames is more pronounced and makes interpolation less accurate.  (Maximum PC created a helpful diagram, pictured right.)

Soap operas are shot on interlaced video at 30fps, and thus represent the epitome cheap video that also strives to look somewhat cinematic.  Motion interpolation artificially increases the frame rate, so we associate the effect with the cheap video look of soap operas.

Shooting at 48fps is something entirely different though.  We will probably associate anything above 30fps with the look of video for some time.  But there is nothing cheap or artificial about 48fps—it doubles the visual information in the presentation by adding frames that were captured in the source footage.  HFR is entirely true to life, and by definition more true to life than 24fps.

 

Too Realistic

“As it turns out, it’s possible for an image to look so clear that it no longer looks real.  Or so real that it takes you out of the film.  As in: that film set looks like…a film set.” -Mike Ryan, Huffington Post

ian-mckellen-cate-blanchett-the-hobbitThe other chief complaint about 48fps is that it shows too much detail—that it looks almost too real.  HFR adds visual information laterally, between frames, so we should not see any individual frame in greater detail.  But since HFR reduces motion blur, the result is a crisper moving image across the frames.  In fact, HFR produces such a crystal clear representation of what was filmed that some say it transforms the theater screen into a window.  Even some of the harshest critics concede, “I can see this working for animation, sports, and nature films.” But this look is not appropriate for the immersive fantasy experience that The Hobbit aims for, or so the argument goes.

That raises the question of what a fantasy film is supposed to look like, and in general what it means to appear cinematic.  The Hobbit was not the ideal test case for 48fps—not because it is a fantasy film, but because it is set in a very familiar fantastical world.  We know Jackson’s Middle-earth well after spending countless hours with the Lord of the Rings trilogy.  But Lord of the Rings was shot on 35mm film, in 2D, at 24fps.  The transition to digital, 3D, and 48fps all at once is understandably jarring.  Such a drastic change is hard to accept, and when the general critical consensus says the story is lacking, it is easy to point to the brand-new HFR technology as a key culprit for the perceived decline in quality from Lord of the Rings to The Hobbit. [1]

the-hobbit-an-unexpected-journey-stone-giantI believe the notion that 24fps is intrinsically cinematic is a fallacy.  The look of 24fps is “cinematic” because it was used in feature films for the last century, but we should not assume 24fps was used because it defines the cinematic ideal.  Film is still a relatively young art form, one that was created by technology, and so we are still experimenting to define what film can be.  Sound in the 20s, color in the 30s, widescreen in the 50s, surround sound in the 70s, CGI in the 90s, digital in the Aughts.  Today we are playing with IMAX, 3D, 4K projections, and finally, frame rates.

So right now, we associate HFR with the high-definition clarity of sports and documentaries.  Given the unstoppable transition from film to digital, that association need not be permanent.  Jackson wants to provide the ultimate immersive fantasy experience, but in 2012, 48fps proved to be more barrier than tool in accomplishing this goal.  Maybe filmmakers will be forced to change the way they design sets, props, costumes, and makeup, because there is less blur to hide the illusion.  If so, they will find a way, and Jackson can properly invite you into a fantasy world even without all that motion blur. [2]

Of course, the success of 48fps depends entirely on our ability to adjust to a higher frame rate.

 

Can We Get Used to It?

Movieline posted an article recently that boldly claimed to be “The Science of High Frame Rates, Or: Why The Hobbit Looks Bad At 48fps.” In the piece, filmmaker James Kerwin proposed a theory based in quantum consciousness:

visual cortex

“Most researchers agree we perceive 40 conscious moments per second.  In other words: our eyes see more than that but we’re only aware of 40.  So if a frame rate hits or exceeds 40fps, it looks to us like reality.  Whereas if it’s significantly below that, like 24fps or even 30fps, there’s a separation, there’s a difference—and we know immediately that what we’re watching is not real… [Some say] you watch it long enough and you won’t associate it with cheap soap operas anymore.  That’s nonsense.  The science does not say that.  It’s not learned behavior.  It’s an inherent part of the way our brains see things.”

I was dubious, as you should be when anyone definitively states the human mind can only perceive X frames per second. [3]  Vulture contacted Rob Allison, a specialist in human perceptual responses at York University, who denied such a limit:

“We don’t perceive at 24 frames per second.  We don’t think at 24 frames per second.  I’ve seen silly things on the Internet about how we think at 40 frames per second.  But there is no perception limit.”

Tim J.  Smith, an expert in visual cognition at Birkbeck University, confirmed to Tested:

“I think it is quite a stretch to go from cortical electrical activity to phenomenological experience of time…  The light reflected from a real-visual scene hits our retinas as a continuous stream (except when we blink or move our eyes), not a discrete series of frames as in film so no increase in frame rate will ever exceed the presentation rate of reality.”

brain movie projectionThe human brain is perfectly capable of processing the infinite “frames” of continuous motion in real life, so there should be no physical issue with adapting to 48fps.  Rather, we have trained ourselves over several decades to expect 24fps (and the associated artifacts) when we go to the theater.  I asked Michael J. Berry II, associate professor at the Princeton Neuroscience Institute, if we can likewise train ourselves to expect 48fps.  Berry responded:

“Yes, I think that we would adapt.  Our visual systems are clearly capable of handling higher frame rates, as all stroboscopically delivered visual stimuli will have less information than real images in the world, which we can clearly process.  So the issue is really one of ‘familiarity’, in the sense of having an expectation of what movies ‘should’ look like.”

This view is supported by Marty Banks, a professor of optometry and vision science at UC Berkeley, who explained to Vulture:

“Colors are not as rich on TV sets or desktop monitors.  The motion is not as smooth.  Brightness is not as great.  We’ve adjusted to that, and we accept it.  So when we see a projection that is closer to reality than what we are used to, our brains go, ‘Whoa!’ and we get this ‘hyper-real’ sensation.  I think that’s what’s happening to people.  Things look unusually sharp [in The Hobbit], and we’re seeing something closer to reality.  If people are frustrated, it’s because this seems a violation of what has been considered normal.  Look at how people reacted when color was first introduced, or Hi-Definition, or Dolby sound.  There were negative reactions to all of those at first, too.  But now if you don’t have them, you feel deprived.”

So the good news is, yes, we can adapt.  Never discount the brain’s astonishing plasticity.  It simply may take time.

 

Should We Get Used to It?

peter jackson hobbit 3d glassesIn April, Jackson estimated we could adjust to 48fps by watching the first “10 minutes or so” of the movie.  Based on anecdotal evidence, it clearly takes more than ten minutes for many.  Allison is more conservative: “By your fourth or fifth high-frame-rate film, you’ll be used to it.” This is in line with Berry’s supposition that “the process of becoming familiar with 48fps movies will presumably unfold over many viewings of such movies.” Although he warns, “I expect it to be inhibited somewhat by continued viewing of 24fps movies, which strengthen our current sense of what a ‘normal’ movie looks like.”

That is a crucial point for the future of 48fps.  Unlike 3D, there is no surcharge for HFR screenings, so studios and theaters have no direct financial incentive to push 48fps.  Warner Bros. opted for a limited release of The Hobbit in HFR 3D to test out the waters.  Based on how well those HFR screenings sell, Warner Bros. will decide whether to expand to more theaters for the Hobbit sequels, set for release in December 2013 and July 2014, respectively.  Three movies over two years may not be enough exposure to adjust to 48fps, but if the audience—voting with their dollar—still largely rejects the technology, HFR could stall again as it did in the 1970s despite Trumbull’s efforts. [4]

“Riot” was a justifiable reason for art critics and music-lovers to bury The Rite of Spring and return to more familiar melodies.  The turning point for The Rite of Spring was that second run of shows, when a curious audience went to the theater with an open mind, and found the beauty in Stravinsky’s work.  The success of The Rite of Spring allowed other composers to experiment the way Stravinsky had, which opened up a new path in the evolution of classical music.

peter jackson hobbit slate

There will be drawbacks if we convert cinema to 48fps.  Jackson noted in his introductory Facebook post:

“We’ve actually become used to [48fps] now, to the point that other film experiences look a little primitive.  I saw a new movie in the cinema on Sunday and I kept getting distracted by the juddery panning and blurring.  We’re getting spoilt!”

Devin Faraci of Badass Digest raises a valid point: If familiarization with 48fps renders the 24fps viewing experience unpleasant, this could be yet another barrier between future generations and a century of classic movies.  He argues:

“I’m not stupid—older movies are always a tough sell to new generations.  The films of the 80s and 90s look impossibly dated to kids born after Kurt Cobain’s suicide.  But widespread adoption of HFR is only going to make that divide much more serious, much deeper.  There are older films like The Wizard of Oz or Lawrence of Arabia that, even when seen with modern sensibilities, work for modern audiences.  Imagine a future where it isn’t just the aesthetics that alienate, but the very presentation format.  It seems to me that such a future is one with a very clear dividing line, as clear as the division between silents and talkies—on one side are the movies people see, and on the other side is 100-plus years of cinema history.”

the wizard of ozWe are still years away from a complete transformation to 48fps if it comes, so it remains to be seen if 24fps will eventually look strange to all of us.  Even the unlikely worst case scenario, where 24fps strains our eyes every time we see it, does not justify rejection of 48fps.  A movie must always be viewed in the context of its time period, whether it is silent, black and white, marked by an old-fashioned acting style, or filled with crude effects.  Maybe one day a low frame rate will be added to the list of considerations.  It will be worth it to accomplish the next logical evolution of film technology with all the promised benefits of 48fps.  When in doubt, do not resist the new to protect the old.

Critics are fighting for the art they love as they know it, and I respect that.  But this is not an issue we can judge solely based on our own first subjective experience.  Presentation in 24fps has treated us very well over the years, but 48fps offers something even greater.  It is our responsibility as cinephiles to give it a chance.  See The Hobbit in 48fps.  See the sequels in 48fps.  Keep an open mind and try to see what Peter Jackson sees.

Footnotes

[1] Tangent: There are plenty of examples where our unconscious makes decisions and we try to rationalize our choice later.  (Film Crit Hulk addressed this idea in the field of film criticism with his great essay on “tangible details.”) I suspect that is part of what is happening here.  HFR is noticeably different.  Our subconscious knows it doesn’t like this difference, but we don’t immediately consciously understand why.  When it comes time to explain why we don’t like HFR, we latch on to familiar experiences (soap operas, behind-the-scenes documentaries) to justify our position.

[2] To be clear, the Hobbit crew was very conscious of 48fps for all elements of the production design.  FX Guide has a great in-depth behind-the-scenes feature that highlights how Weta handled all the changes from Lord of the Rings.  Visual effects supervisor Eric Saindon notes, “We really went over the top and scanned every single set that it was shot on, which was absolutely beneficial.  We had 3D scans of everything—we knew the placement of everything when we got it back in the 3D world.  Because everything’s in stereo and at 48fps we needed a lot more detail.”

[3] Jackson himself states, “Science tells us that the human eye stops seeing individual pictures at about 55fps.” I have no idea how he came to this conclusion.  My best guess says he is basing that on the critical flicker fusion.  The graphs below from Camera Technica suggest that the CFF tops out around 45-60hz based on the brightness, area, and distance of the image, respectively.  But this estimate would not hold for all viewing experiences.  And regardless, the issue of frame rate perception is more complicated than just looking at CFF, explained well in this detailed guide.

cff graph luminance

cff graph distance

cff graph area

[4] Okay, that’s a bit dramatic.  James Cameron, the most powerful director in the business, is intent on shooting the sequels to Avatar, the highest grossing movie of all time, in 48fps.  Even if the Hobbit HFR experiment is deemed a failure, Cameron is the best backup plan a technology can have.  But without the financial incentive of a surcharge, studios will not push 48fps on the industry if there is no demand for it.

 

[Edit: A previous version of this article mistakenly credited Jonah Lehrer as a neuroscientist.]

Latest News

Pages: 1 2 3

Close