A lot of people are concerned with political polarization–the idea that people are becoming divided into opposing camps that don’t communicate with each other. (Not everyone worries about polarization–some people see it as healthy political competition, others worry about opposite problems such as political apathy and non-participation, and you even used to hear people say that there wasn’t a dime’s worth of difference between the two parties.)

Anyway, polarization can be measured in various ways: one approach is to ask people who they talk with, and find out the extent to which people mostly associate with people similar to them. Another method is to look at people’s positions on the issues and see if most people have extreme positions. Regarding this latter approach, Jeremy Freese points out a potential source of measurement bias:

Occasionally social scientists become interested in whether Americans are becoming “more polarized” in their opinions. The obvious strategy for considering this question is to take a bunch of survey items that have been asked of comparable samples in the past and now, and to look at whether people hold more divergent views now than they did then. . . . [But] people buying survey time are typically interested in questions that vary. If they are asking a question that doesn’t vary, it’s for some reason, like perhaps because it has been asked repeatedly in the past. . . . So, items that would provide evidence of polarization — consensus then, divergence now — are disproportionately less likely to be part of the universe of available items for comparison over time, while items that provide evidence of no polarization — divergence then, consensus now — are disproportionately more likely. And thus researchers claim to producing findings about the world of public opinion when the patterns in their data actually reflect the world of public opinion surveys.

It’s an interesting issue–selection bias of questions rather than the usual worries about survey respondents–and it’s something that Delia and I thought about some when using National Election Studies to analyze trends in issue polarization. These issues are real, although I don’t know that it’s such a problem as you say, because in any case the inferences will be conditional on whatever questions you happen to be studying–so, in any case, the researcher has to justify which issues he or she is looking at.

It happens all the time

Here I just want to point out that these measurement issues are not unique to the study of polarization. For example, is the Supreme Court drifting to the left, the right, or roughly staying the same? These things can be measured, but with difficulty because it depends on the docket for each year.

Or you could even ask simpler questions about median voters. For example, when I wrote why it it can be rational to vote (because you can feel that having your preferred candidate win would likely make a big difference to many millions of people), some people replied that it’s somewhat naive to feel that _your_ preferred candidate will be so great: if approximately half the people preferred Bush and half preferred Kerry, then what makes you so sure that your views are more valid than the other 50% of the population? One difficulty with that argument is that the answer depends on the reference set. For example, suppose you live in Texas. If you voted for Kerry, who are you to say that your judgment is better than the 61% who supported Bush? On the other hand, if you voted for Bush, who are you to say that your judgment is better than the (presumably) vast majority of people around the world who hate the guy? What it means to be in the “center” depends on your reference set. I’m sure there are many other examples of this sort of selection bias in measurements.

Back to polarization

The way that Delia and I actually measured polarization was through correlations between issue attitudes. The idea is that, if the population is becoming more polarized, this should show up as increasing coherence in issue positions, so that if I know where you stand on abortion, I’m more likely to be able to figure out where you stand on social security (for example). You can see our results here: they seem consistent with Fiorina’s theory that voters are sorting into parties more than they are polarizing on the issues.

One other amusing thing (well, it’s amusing if you’re a statistician, maybe)

Polarization is a property of a population, not of individuals. It doesn’t mean anything (usually) to say that “I am polarized” but you can talk about a group of people being polarized into different subgroups, or polarized along some dimensions. The polarization of a group cannot be expressed as a sum or average of polarizations of individuals. It’s an interesting example, because many (most?) of the things we measure in this way tend to be individual properties that we simply aggregate up (for example, the percentage of people who support candidate X, or the average age of people in a group, or whatever). In statistical terms, polarization is a property of the distribution, not of the random variable.