Cinemath: Good Movies Do Better at the Box Office; Also Helps to Be Expensive, PG-13, and a Sequel

     June 17, 2012

In case you found this page first, click here to read the first page of this article.


This section is mostly going to be an information dump that will only be of interest to the 1% of the population that uses Minitab and/or R. (Hey, anyone want to teach me SAS?) I add a bit more commentary, and wanted to include this section to support the analysis. But the heavy lifting of the interpretation is left to you.

First, here is the Minitab output for both the 2011 and 2010 data sets. Notice that there is a higher coefficient for RT and a higher R when the regression model is based on the 2010 dataset.


box office reviews statistics


box office reviews statistics

Of course, these models don’t satisfy the necessary assumptions. There is no multicollinearity, and there is sufficient linearity. But the normality assumption is probably violated and there is definite heteroscedasticity.

To counteract this, a transformation of the dependent variable Y^(1/3) is needed. This transformation yields a better fit, but the transformation is much harder to explain than a one-unit increase on both sides of the equation, so I stuck with the basic regression model on the first page. There is some benefit to using polynomial regression, especially with the theater count. But since this model is a good fit without raising anything else to a power, I left it at this.

The Minitab output and relevant plots for the transformed regression model follows.

box office reviews statistics

box office reviews statistics

box office reviews statistics

box office reviews statistics

  • Lizard King

    Brendan, I always look forward to your Cinemath articles. Possibly one of my favorite things to read on the entire internet. Informative, well explained, interesting, and unique. Each article has been fantastic. Keep up the awesome work!

    •!/colliderbrendan Brendan Bettinger

      Thanks! Wish I could do them more frequently.

  • Anarvin

    Great read as usually Brendan! : )

  • Tim

    In my opinion, Rotten Tomatoes is the worst thing to happen to cinema. People now see a movie based on what others say instead of forming their own opinion. Sad. For example: I hated Iron Man, but it got a fresh rating.

    • D. McHugh

      I don’t know. I used to take RT much more seriously when it was just the Top Critics of major media outlets. Now, it seems any Joe Blow with a computer in his bedroom can sign up and review movies. These opinions from so many random people from non-exiistant or self-created websites can easily drag down the overall rating of a great film while elevating a mediocre one. While I never see a film solely based on any one person’s opinion, I still consider what my go-to Big 3 say: Roger Ebert (Chicago Times), Peter Travers (Rolling Stone) and Richard Roeper.

      • Clay

        My top 3 are Ebert, Michael Phillips and James Berardinelli.

      • Alex

        While it does seem that the availability is there for just “anyone” to be able to submit to RT, there are certain guidelines that demand a certain level of quality of writing, consistency and time being a reviewer/number of reviews written. It is not as easy as it seems.

    • drod

      I know what you mean, while I loved Iron Man I tend to disagree with critics for example Public Enemies is my favorite movie and it only received a 69% on RT.

      • Donovan McLean

        Which is fresh.

    • Max

      Not relevant at all to what you’re saying, but most people love Iron Man

  • David

    Awesome work. As someone who worked in mathematics at the college level, I’m impressed. And I just had to test the formula myself. The Social Network grossed around $96 million, and the formula gives around $91 mil. I always find myself laughing at box office forecasts – one “widely trusted” website had That’s My Boy opening for twenty million this weekend. I knew that was a joke when I first saw it. It would be an interesting follow-up to discover a more accurate opening weekend predictor.

  • paul tracy

    ironically, collider pushes the hype muscle machine trash. but, this is at least a positive step in the right direction.

    :: next on collider, perfumes inspired by the dark knight rises ::

    • Konrad-ko-man-der

      It’s called Sex Panther by Odeon. It’s illegal in nine countries… Yep, it’s made with bits of real panther, so you know it’s good.

      • paul tracy

        holy shit that made me laugh.

        nicely done.

        ~ pp

  • Ricky Johnston

    Is there anyway you can share the data set. I would love to give this as a homework assignment to my class

    •!/colliderbrendan Brendan Bettinger

      Sure. Will you get in touch with me: colliderbrendan at gmail dot com.

  • t

    I didn’t understand a word of this article but it spoke to me.

    • Ray

      OMG. Me Too. I’m Like Huh but I do know the Man speaks the Truth.

      All Honestly. I wanna see a movie. I’ll will see no matter what critics says. I respect their opinions though.

  • enzo

    If all critics gave a movie 51% it would have a 100% value on RT. Metacritic is a better service using the actual average score with prominent critics carrying more weight.

    I would have liked to have seen Brenden’s numbers using metacritic, but still, the premise that critics evaluations actually count for something is reassuring.

    Other variables: Genre inclination, Star/Actor and Director prominence. Critics always cut slack for a Scorsese or de Niro flick. And documentaries always seem to score higher while comedies score lower.

    •!/colliderbrendan Brendan Bettinger

      Metacritic introduces their own bias in trying to assign a number to every review, especially since each critic uses a different scale. Then they weight the average according to perceived importance. Plus, they have smaller sample sizes—the Piranha 3DD is based on just 13 critics.

      RT only needs to determine whether a review is positive or negative (admittedly with some error). And they have larger sample sizes (compare the 43 reviews for Piranha 3DD). I don’t trust Metacritic, and prefer RT for statistical analysis.

      • Bonobo

        Rotten Tomatoes also has an “Avarage Rating” for each movie, where they do basically the same thing as metacritic, but from their larger sample size. I think it would be interesting to see how that rating fits into your model, perhaps alongside the rotten/fresh rating.

        The RT Average Rating is a number I personally find more telling than the tomatometer, and both of them together give a much clearer indication of the critical response. Sometimes a 90%+ on RT can just mean that everyone agrees that a particular movie is mildly entertaining.

        Take Ben Affleck’s The Town for instance. A solid but unspectacular thriller which has a puzzling 94% on the Tomatometer, but a much more reasonable 7,8 average rating. Compare that to Eternal Sunshine of the Spotless Mind, an original and fascinating film, which has a 93% on the Tomatometer, but a much stronger 8.4 average rating.

        I think people often have undue expectations of films with high RT scores, as they equate such scores with high praise, not universal (but perhaps unenthusiastic) praise. This causes them to loose faith in the validity of Rotten Tomatoes (I know a few such people myself), so I really hope they will play up the Avarage Rating a bit.

  • Armand

    If anyone has been paying attention to the highest grossing movies of each year, they’d notice that it’s been a great one for the last 5 years! Starting with 08′ The Dark Knight, 09′ Avatar, 10′ Toy Story, 11′ Harry Potter 7-2, 12′ – As great as the Dark Knight Rises could be, the Avengers is destined to be the top grossing movie of the year, still a great movie.

    • Arnold

      U forgot about THE HOBBIT.

      • Armand

        The Hobbit is going to be a great movie, but it wont beat the Avengers at the Box Office, seeing as its part one, those who feel cheated by the story being split two (who are complete morons) may just skip it in the theaters, rent it, and just go see part 2, just my opinion though, anything could happen

  • Lance

    A thoughtful bit of analysis, and the results are definitely interesting. Thanks for posting this, Brendan!

  • John M.

    Love the math. Trendlines are truth!

  • Anarvin

    Am I just bad at math, or the formula doesnt apply for almost any of this years releases?

  • J

    This is a really cool idea and I do think there is a correlation to RT% and actual gross but I dont see any way it can work with a formula. For example, Dark Shadows comes out to 132 million and its actual domestic gross was 74 million. That could have been because of Burtons recent films disappointing and the amount of competition (two factors that cant be broken down to a number). Hunger Games is not a sequel but it should be treated like one because of its large fanbase. Regardless, the formula gives 133 million and the domestic gross just crossed 400 million. There are movies like Paranormal Activity, where the budget (or lack thereof) has no bearing on the gross. Unless 3D charges are added in, it cant calculate any movie that charges an extra $3. Marketing is generally not released to the public but it can make a movie a megahit (Avengers) or even more of a flop (Green Lantern). The number of theaters is a worldwide number yet the formula is only calculating domestic. I love RT and trust it to determine if I should spend $10 or $1, but there are far too many other factors to definitively say there is a connection. This is an excellent and ambitious article so i am impressed, but i think it can be improved (but not perfected).

    •!/colliderbrendan Brendan Bettinger

      I tried to address most of your concerns in the text, and there is a better model in the appendix. But yes, you can always find examples that succeed or fail on factors I can’t account for.

      Number of theaters is domestic. I would have preferred to look at the worldwide gross, but did not because I only had information on domestic theater counts.

  • Pingback: La formula matematica per calcolare in anticipo il successo di un film « News «()

  • Jed D

    The studios have internally been using linear regression for years to “predict” box office (as well as ancillary markets). There are several problems in predicting box office. One is causality (the number of theaters doesn’t cause higher box office, but it’s correlated because, in order to get higher box office, you have to have a large number of theaters). Another is that the RT score isn’t available when you are trying to decide how much to spend to market the film, let alone when you greenlight the picture.

    What jumps out at me, eyeballing the scatter plots, is that if you limit the analysis to the “hits” (the movies that pay the bills) — say films with box office of $150 million or greater, the trend line looks like it would be either flat or is possibly negatively correlated with the RT score. And the R2 measure looks like it would be will well under .5.

  • Andy

    Assume the studios fib about the budgets by around 25%. For many years now, that’s what they figure they can get away with without being called out-and-out liars. When the budget is under $15 million, they will, however tell the truth because that’s when they get concessions from the unions.

  • sofasobad

    Brendan, why did you only look at movies that played in more than 600 theatres? Don’t some of the best-reviewed prestige arthouse films play in far fewer theatres than that? Doesn’t setting a high limit on the theatre count skew the analysis?

  • Pingback: 21 Jump Street | I Hate Your Favourite Movie()

  • Pingback: 2012: Year Of The Female Action Hero? | USA Press()

  • Pingback: 2012: Year Of The Female Action Hero? - US Mag - The Breaking News | US Mag - The Breaking News()

  • Pingback: 2012: Year Of The Female Action Hero? |

  • Pingback: 2012: Year Of The Female Action Hero? | Simply Boundless Entertainment()

  • Pingback: Good Movies Actually Do Better at the Box Office | Gear Diary()

  • Pingback: Good Movies Actually Do Better At The Box Office()

  • Pingback: Cinemath: 2013 Box Office Stats. A Year-End Report on the 2013 Box Office | Collider()

Pages: 1 2