Critics’ conformity to consumers in movie evaluation

Holbrook (1999) contrasts (i) “elite” culture with “pop” culture and (ii) “popular” taste with “legitimate” taste. Conflating elite culture and legitimate taste itself reflects elitism by suggesting that popular taste is less legitimate. In this paper, by “elite taste” we mean aesthetic standards held by a minority with specialized education, training, or experience in movie criticism. We use “popular taste” to mean a focus on entertainment standards held by the majority, who do not have such specialized education, training, or experience.

To clarify the distinction between professional critics and social media influencers, we searched all critics in our data (n = 450) to confirm that they work for at least one media outlet as professional movie critics. We also searched their social media activities on Twitter and Instagram (see Web Appendix A). These results indicate that 83.11% of critics have a Twitter account and 40.44% have an Instagram account. Of critics using Twitter, 75.13% had fewer than 10,000 followers and 58.56% posted fewer than 1000 tweets per year. Among critics using Instagram, 93.96% had fewer than 10,000 followers and 96.70% published fewer than 1000 posts per year. This suggests that a relatively small proportion of critics in our data might be viewed as social media influencers, but in any case all critics are professionals working for one or more media outlet.

We did not use Metacritic’s Metascore since the weight given to critics based on Metacritic’s judgment of review quality and overall stature raise concerns about transparency, and the score is updated in real time, so there is no average daily score.

We performed a Granger test for the effect of cumulative consumer ratings on critic ratings, and found F = 69.78 with p < 0.01. This result suggests that the previous consumer ratings could explain the variation in current critic ratings, consistent with our theory and supporting the validity of our model specification.

We used production but not advertising budget to avoid multicollinearity problems. The pre-release advertising budget is typically a fixed proportion of the production budget (Vogel, 2007) and pre-release advertising accounts for the vast majority of advertising spending (e.g., 88%, Elberse, 2007).

Following Chintagunta et al. (2010), we did not treat variances of critic and consumer ratings as endogenous because potential sources of the correlations between the variances of both critic and consumer ratings and the error term were unknown, making it less clear what appropriate instruments for the variances would be.

Rotten Tomatoes, a popular movie website with critic and consumer reviews, differs from IMDb in two respects. First, Rotten Tomatoes encourages critics to upload reviews on the website; IMDb asks critics to submit links to their reviews and shows only the links. Second, Rotten Tomatoes establishes criteria for critics and only approved critics can upload reviews to the site while IMDb only performs a post hoc check and reserves the right to reject submitted links. Rotten Tomatoes also differs from Metacritic in having a much longer list of approved critics (about 3000), because they use different criteria to screen critics.

Note that once the interactions were added to the model, the main effect of volume of consumer ratings turned negative and significant. We drew a margin plot in65to illustrate the simultaneous interplay of the main effect of consumer ratings, the main effect of volume of consumer reviews, and their interactions.

We performed more robustness checks by adding fixed effects for movies, critics, and media. Most of the robustness checks produced results consistent with the previous ones (see Web Appendix F), except when the model included both movie and critic fixed effects. One possible explanation for this exceptional case is overfitting, which occurs if observation-specific individual fixed effects (IFE) are specified. This is exactly the case when we added both critic and movie fixed effects to the model, as each critic only wrote one review for each movie. As noted by Wolf and Ritschl (2011), “Overfitting in treatment effect regressions through redundant fixed effects causes downward bias in the standard error of the estimated treatment effect. This bias is roughly proportional to the reduction in the residual sum of squares introduced by overfitting” (p. 297). In our case, when we added both movie and critic fixed effects into our model, the F value was 0.699 and the corresponding p value was 0.711, which is a clear sign of overfitting.

Most studies in the movie literature did not directly report the elasticities because they did not use log transformation of critic/consumer ratings (e.g., Liu, 2006; Chintagunta et al., 2010). Among those using log-log models, Duan et al. (2008a, 2008b) reported elasticity of consumer ratings on box office revenue to be 0.43*** (based on the OLS results); Elberse and Eliashberg (2003) reported elasticity of critic ratings on box office revenue to be 0.55** (based on the OLS results of the USA data). No log-log models were found for the effect of critic ratings on consumer ratings. Yet, from the remaining studies that we reviewed, we can infer from their model estimates that all three elasticities are positive.