Measuring media bias

Tim Groseclose and Jeffrey Milyo wrote a paper on "A measure of media bias." Here's the paper, and here's the abstract to the paper:

We [Groseclose and Milyo] measure media bias by estimating ideological scores for several major media outlets. To compute this, we count the times that a particular media outlet cites various think tanks and policy groups, then compare this with the times that members of Congress cite the same groups. Our results show a strong liberal bias: all of the news outlets we examine, except Fox News’ Special Report and the Washington Times, received scores to the left of the average member of Congress. Consistent with claims made by conservative critics, CBS Evening News and the New York Times received scores far to the left of center. The most centrist media outlets were PBS NewsHour, CNN’s Newsnight, and ABC’s Good Morning America; among print outlets, USAToday was closest to the center. All of our findings refer strictly to news content; that is, we exclude editorials, letters, and the like.

They fit a version of an ideal-point model to mentions of "think tanks and policy groups" by Congressmembers and media outlets and they find, basically, that most of the newspapers they look at quote a mixture of groups that is similar to moderate-to-conservative Democrats, most of the TV shows are comparable in quotations to conservative Democrats in Congress, and two of the more partisan Republican news organizations (Fox News and the Washington Times) have quotation patterns that are comparable to liberal Republicans in Congress.

This makes sense--as the authors note, surveys have found that many more journalists are Democrats than Republicans, but partisans of both sides have to moderate their views in order to maintain journalistic credibility.

I wonder to what extent these bias measures depend on the issues under consideration. There's also the question of the relevance of quotation patterns to the larger questions of bias. As well as the question of what role the press should be expected to take in a representative democracy--for example, will a mass-readership press be expected to hold more left-wing views so as to be popular with readers, or more right-wing views so as to be popular with advertisers? There's also the difference between local and national news. Lots to think about.

Brendan's comments

P.S. Here are Brendan Nyhan's thoughts criciticm of the paper. Brendan's criticisms seem valid to me; notheless I'm a bit more positive than Brendan is about the paper, I think because the problem of studying media bias is tough, and I'm impressed about what Groseclose and Milyo did manage to do. Perhaps just my own bias in showing an affinity with quantitative researchers . . . I do agree, though, that "bias" isn't quite the right word to discuss what Groseclose and Milyo measure, since "bias" implies a deviation from some unbiased position or truth, which I don't see them measuring.

P.S.

I also have a few comments on the presentation of the results:

bullfighter

Too bad your endorsement of Nyhan's critique is so tepid. It is even more puzzling given that you realize that "bias" is a misnomer for what G&M are measuring. Such terminology makes their paper positively misleading. And there are flaws in their reasoning that Nyhan hasn't even addressed; for example, even if the U.S. population as a whole were (say, by definition) unbiased, there is a big leap from measuring the median position of the members of Congress to asserting that it equals the median position of voting population to equating that to the median position of the whole population.

Statistical Modeling, Causal Inference, and Social Science: Measuring media bias

Measuring media bias

bullfighter